Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_mpnn_from_auc/config.json --data_path /home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/train_full.csv',
 'dataset_type': 'classification',
 'depth': 2,
 'device': device(type='cuda', index=1),
 'dropout': 0.35000000000000003,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': False,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 1900,
 'ffn_num_layers': 2,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 1,
 'grad_clip': None,
 'hidden_size': 1900,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_avg_feats_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_avg/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/rgb_nfs/dsets/cov_2_cl/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.577934
Epoch 1
Validation auc = 0.702582
Epoch 2
Validation auc = 0.691549
Epoch 3
Validation auc = 0.718310
Epoch 4
Loss = 4.4896e+01, PNorm = 78.6973, GNorm = 1621.7398, lr_0 = 6.5000e-04
Validation auc = 0.699765
Epoch 5
Validation auc = 0.702347
Epoch 6
Validation auc = 0.715023
Epoch 7
Validation auc = 0.692254
Epoch 8
Validation auc = 0.651174
Epoch 9
Loss = 7.7895e+01, PNorm = 78.9902, GNorm = 3434.2381, lr_0 = 9.9743e-04
Validation auc = 0.669953
Epoch 10
Validation auc = 0.541784
Epoch 11
Validation auc = 0.644131
Epoch 12
Validation auc = 0.752113
Epoch 13
Validation auc = 0.692019
Epoch 14
Loss = 4.6324e+01, PNorm = 79.2988, GNorm = 864.1287, lr_0 = 9.8890e-04
Validation auc = 0.623474
Epoch 15
Validation auc = 0.665962
Epoch 16
Validation auc = 0.619484
Epoch 17
Validation auc = 0.684038
Epoch 18
Validation auc = 0.547418
Epoch 19
Loss = 4.3965e+01, PNorm = 79.5137, GNorm = 1492.9600, lr_0 = 9.8045e-04
Validation auc = 0.546009
Epoch 20
Validation auc = 0.679343
Epoch 21
Validation auc = 0.693192
Epoch 22
Validation auc = 0.665258
Epoch 23
Validation auc = 0.616667
Epoch 24
Loss = 5.3357e+01, PNorm = 79.6574, GNorm = 1479.4173, lr_0 = 9.7207e-04
Validation auc = 0.638498
Epoch 25
Validation auc = 0.640141
Epoch 26
Validation auc = 0.616901
Epoch 27
Validation auc = 0.662676
Epoch 28
Validation auc = 0.629343
Epoch 29
Loss = 4.1960e+01, PNorm = 79.7657, GNorm = 703.4413, lr_0 = 9.6376e-04
Validation auc = 0.713615
Epoch 30
Validation auc = 0.751174
Epoch 31
Validation auc = 0.703521
Epoch 32
Validation auc = 0.653756
Epoch 33
Validation auc = 0.654695
Epoch 34
Loss = 4.3170e+01, PNorm = 79.8672, GNorm = 469.5237, lr_0 = 9.5552e-04
Validation auc = 0.709624
Epoch 35
Validation auc = 0.700939
Epoch 36
Validation auc = 0.717840
Epoch 37
Validation auc = 0.744131
Epoch 38
Validation auc = 0.701878
Epoch 39
Loss = 2.9226e+01, PNorm = 79.9937, GNorm = 680.7071, lr_0 = 9.4735e-04
Validation auc = 0.698122
Epoch 40
Validation auc = 0.719249
Epoch 41
Validation auc = 0.684038
Epoch 42
Validation auc = 0.651174
Epoch 43
Validation auc = 0.634272
Epoch 44
Loss = 2.1170e+01, PNorm = 80.1301, GNorm = 460.1611, lr_0 = 9.3925e-04
Validation auc = 0.657746
Epoch 45
Validation auc = 0.672770
Epoch 46
Validation auc = 0.665728
Epoch 47
Validation auc = 0.686854
Epoch 48
Validation auc = 0.699765
Epoch 49
Loss = 2.9292e+01, PNorm = 80.3213, GNorm = 584.0912, lr_0 = 9.3122e-04
Validation auc = 0.711268
Epoch 50
Validation auc = 0.667606
Epoch 51
Validation auc = 0.714085
Epoch 52
Validation auc = 0.706573
Epoch 53
Validation auc = 0.667371
Epoch 54
Loss = 2.5514e+01, PNorm = 80.5163, GNorm = 674.6844, lr_0 = 9.2326e-04
Validation auc = 0.634742
Epoch 55
Validation auc = 0.634272
Epoch 56
Validation auc = 0.705164
Epoch 57
Validation auc = 0.715962
Epoch 58
Validation auc = 0.721362
Epoch 59
Loss = 2.2962e+01, PNorm = 80.7620, GNorm = 879.3186, lr_0 = 9.1537e-04
Validation auc = 0.655164
Epoch 60
Validation auc = 0.691080
Epoch 61
Validation auc = 0.748592
Epoch 62
Validation auc = 0.733568
Epoch 63
Validation auc = 0.700939
Epoch 64
Loss = 1.1088e+01, PNorm = 81.0922, GNorm = 534.7043, lr_0 = 9.0754e-04
Validation auc = 0.701174
Epoch 65
Validation auc = 0.734507
Epoch 66
Validation auc = 0.764789
Epoch 67
Validation auc = 0.726291
Epoch 68
Validation auc = 0.661268
Epoch 69
Loss = 1.4625e+01, PNorm = 81.5244, GNorm = 573.3432, lr_0 = 8.9978e-04
Validation auc = 0.683099
Epoch 70
Validation auc = 0.747183
Epoch 71
Validation auc = 0.749765
Epoch 72
Validation auc = 0.714554
Epoch 73
Validation auc = 0.620892
Epoch 74
Loss = 1.6727e+01, PNorm = 81.8600, GNorm = 523.7399, lr_0 = 8.9209e-04
Validation auc = 0.632864
Epoch 75
Validation auc = 0.713146
Epoch 76
Validation auc = 0.726995
Epoch 77
Validation auc = 0.710329
Epoch 78
Validation auc = 0.688967
Epoch 79
Loss = 9.1510e+00, PNorm = 82.1909, GNorm = 465.7826, lr_0 = 8.8447e-04
Validation auc = 0.671831
Epoch 80
Validation auc = 0.627230
Epoch 81
Validation auc = 0.630986
Epoch 82
Validation auc = 0.646244
Epoch 83
Validation auc = 0.680282
Epoch 84
Loss = 1.0407e+01, PNorm = 82.4927, GNorm = 174.2576, lr_0 = 8.7691e-04
Validation auc = 0.677465
Epoch 85
Validation auc = 0.731690
Epoch 86
Validation auc = 0.742723
Epoch 87
Validation auc = 0.718075
Epoch 88
Validation auc = 0.690141
Epoch 89
Loss = 1.3139e+01, PNorm = 82.9699, GNorm = 413.5107, lr_0 = 8.6941e-04
Validation auc = 0.658920
Epoch 90
Validation auc = 0.706338
Epoch 91
Validation auc = 0.740845
Epoch 92
Validation auc = 0.740376
Epoch 93
Validation auc = 0.697887
Epoch 94
Loss = 1.8022e+01, PNorm = 83.3673, GNorm = 448.4998, lr_0 = 8.6198e-04
Validation auc = 0.633568
Epoch 95
Validation auc = 0.634742
Epoch 96
Validation auc = 0.710798
Epoch 97
Validation auc = 0.688498
Epoch 98
Validation auc = 0.700939
Epoch 99
Loss = 8.6771e+00, PNorm = 83.7225, GNorm = 233.2688, lr_0 = 8.5461e-04
Validation auc = 0.638028
Epoch 100
Validation auc = 0.621596
Epoch 101
Validation auc = 0.648592
Epoch 102
Validation auc = 0.682864
Epoch 103
Validation auc = 0.682629
Epoch 104
Loss = 1.5283e+01, PNorm = 84.0608, GNorm = 619.2325, lr_0 = 8.4730e-04
Validation auc = 0.684977
Epoch 105
Validation auc = 0.676761
Epoch 106
Validation auc = 0.662911
Epoch 107
Validation auc = 0.685446
Epoch 108
Validation auc = 0.713850
Epoch 109
Loss = 7.2022e+00, PNorm = 84.5289, GNorm = 458.2705, lr_0 = 8.4006e-04
Validation auc = 0.734977
Epoch 110
Validation auc = 0.720188
Epoch 111
Validation auc = 0.677700
Epoch 112
Validation auc = 0.666667
Epoch 113
Validation auc = 0.681690
Epoch 114
Loss = 6.9898e+00, PNorm = 84.9099, GNorm = 457.9008, lr_0 = 8.3288e-04
Validation auc = 0.701643
Epoch 115
Validation auc = 0.712676
Epoch 116
Validation auc = 0.715023
Epoch 117
Validation auc = 0.714085
Epoch 118
Validation auc = 0.655399
Epoch 119
Loss = 5.5245e+00, PNorm = 85.1497, GNorm = 230.9498, lr_0 = 8.2576e-04
Validation auc = 0.656573
Epoch 120
Validation auc = 0.701878
Epoch 121
Validation auc = 0.741784
Epoch 122
Validation auc = 0.756338
Epoch 123
Validation auc = 0.756338
Epoch 124
Loss = 5.4689e+00, PNorm = 85.6105, GNorm = 467.6901, lr_0 = 8.1870e-04
Validation auc = 0.770892
Epoch 125
Validation auc = 0.779343
Epoch 126
Validation auc = 0.771127
Epoch 127
Validation auc = 0.765023
Epoch 128
Validation auc = 0.772300
Epoch 129
Loss = 5.1161e+00, PNorm = 86.1388, GNorm = 287.2558, lr_0 = 8.1170e-04
Validation auc = 0.765962
Epoch 130
Validation auc = 0.776526
Epoch 131
Validation auc = 0.767371
Epoch 132
Validation auc = 0.753286
Epoch 133
Validation auc = 0.743427
Epoch 134
Loss = 5.6472e+00, PNorm = 86.4146, GNorm = 224.5757, lr_0 = 8.0476e-04
Validation auc = 0.717371
Epoch 135
Validation auc = 0.694131
Epoch 136
Validation auc = 0.703052
Epoch 137
Validation auc = 0.710094
Epoch 138
Validation auc = 0.712441
Epoch 139
Loss = 3.6549e+00, PNorm = 86.8065, GNorm = 284.7464, lr_0 = 7.9788e-04
Validation auc = 0.711268
Epoch 140
Validation auc = 0.708216
Epoch 141
Validation auc = 0.707277
Epoch 142
Validation auc = 0.701408
Epoch 143
Validation auc = 0.682864
Epoch 144
Loss = 3.9668e+00, PNorm = 87.2004, GNorm = 303.2324, lr_0 = 7.9106e-04
Validation auc = 0.695775
Epoch 145
Validation auc = 0.706808
Epoch 146
Validation auc = 0.709859
Epoch 147
Validation auc = 0.728638
Epoch 148
Validation auc = 0.727700
Epoch 149
Loss = 4.5181e+00, PNorm = 87.5089, GNorm = 247.0539, lr_0 = 7.8430e-04
Validation auc = 0.659624
Epoch 150
Validation auc = 0.649531
Epoch 151
Validation auc = 0.721127
Epoch 152
Validation auc = 0.738498
Epoch 153
Validation auc = 0.720657
Epoch 154
Loss = 5.5125e+00, PNorm = 87.8892, GNorm = 340.0575, lr_0 = 7.7759e-04
Validation auc = 0.709390
Epoch 155
Validation auc = 0.725822
Epoch 156
Validation auc = 0.701878
Epoch 157
Validation auc = 0.697887
Epoch 158
Validation auc = 0.703286
Epoch 159
Loss = 4.0450e+00, PNorm = 88.2209, GNorm = 215.4950, lr_0 = 7.7095e-04
Validation auc = 0.689671
Epoch 160
Validation auc = 0.685446
Epoch 161
Validation auc = 0.687324
Epoch 162
Validation auc = 0.711972
Epoch 163
Validation auc = 0.736150
Epoch 164
Loss = 3.5927e+00, PNorm = 88.5282, GNorm = 200.6164, lr_0 = 7.6436e-04
Validation auc = 0.745775
Epoch 165
Validation auc = 0.736854
Epoch 166
Validation auc = 0.697653
Epoch 167
Validation auc = 0.691080
Epoch 168
Validation auc = 0.717136
Epoch 169
Loss = 2.9517e+00, PNorm = 88.8875, GNorm = 255.8673, lr_0 = 7.5782e-04
Validation auc = 0.765023
Epoch 170
Validation auc = 0.771127
Epoch 171
Validation auc = 0.771362
Epoch 172
Validation auc = 0.754695
Epoch 173
Validation auc = 0.744601
Epoch 174
Loss = 7.8125e+00, PNorm = 89.1796, GNorm = 162.8499, lr_0 = 7.5134e-04
Validation auc = 0.757042
Epoch 175
Validation auc = 0.760563
Epoch 176
Validation auc = 0.766432
Epoch 177
Validation auc = 0.759859
Epoch 178
Validation auc = 0.742723
Epoch 179
Loss = 4.3351e+00, PNorm = 89.4831, GNorm = 192.4846, lr_0 = 7.4492e-04
Validation auc = 0.731455
Epoch 180
Validation auc = 0.722300
Epoch 181
Validation auc = 0.719484
Epoch 182
Validation auc = 0.720892
Epoch 183
Validation auc = 0.709155
Epoch 184
Loss = 4.7638e+00, PNorm = 89.7705, GNorm = 262.8069, lr_0 = 7.3855e-04
Validation auc = 0.717840
Epoch 185
Validation auc = 0.722535
Epoch 186
Validation auc = 0.722535
Epoch 187
Validation auc = 0.728638
Epoch 188
Validation auc = 0.724648
Epoch 189
Loss = 1.5021e+00, PNorm = 90.0682, GNorm = 63.1433, lr_0 = 7.3224e-04
Validation auc = 0.721362
Epoch 190
Validation auc = 0.719718
Epoch 191
Validation auc = 0.700704
Epoch 192
Validation auc = 0.706338
Epoch 193
Validation auc = 0.720423
Epoch 194
Loss = 3.0940e+00, PNorm = 90.3863, GNorm = 163.4283, lr_0 = 7.2598e-04
Validation auc = 0.716667
Epoch 195
Validation auc = 0.711737
Epoch 196
Validation auc = 0.707042
Epoch 197
Validation auc = 0.694131
Epoch 198
Validation auc = 0.678638
Epoch 199
Loss = 1.2055e+00, PNorm = 90.7260, GNorm = 93.7920, lr_0 = 7.1977e-04
Validation auc = 0.656808
Epoch 200
Validation auc = 0.663615
Epoch 201
Validation auc = 0.660563
Epoch 202
Validation auc = 0.667371
Epoch 203
Validation auc = 0.676995
Epoch 204
Loss = 2.5779e+00, PNorm = 91.0245, GNorm = 245.2734, lr_0 = 7.1362e-04
Validation auc = 0.688028
Epoch 205
Validation auc = 0.693192
Epoch 206
Validation auc = 0.694366
Epoch 207
Validation auc = 0.699061
Epoch 208
Validation auc = 0.714554
Epoch 209
Loss = 4.5503e+00, PNorm = 91.3315, GNorm = 262.7706, lr_0 = 7.0752e-04
Validation auc = 0.712911
Epoch 210
Validation auc = 0.709155
Epoch 211
Validation auc = 0.715962
Epoch 212
Validation auc = 0.668075
Epoch 213
Validation auc = 0.622066
Epoch 214
Loss = 3.4267e+00, PNorm = 91.6704, GNorm = 334.3162, lr_0 = 7.0147e-04
Validation auc = 0.615493
Epoch 215
Validation auc = 0.648826
Epoch 216
Validation auc = 0.707981
Epoch 217
Validation auc = 0.719484
Epoch 218
Validation auc = 0.724883
Epoch 219
Loss = 2.1562e+00, PNorm = 91.9479, GNorm = 148.9232, lr_0 = 6.9548e-04
Validation auc = 0.743427
Epoch 220
Validation auc = 0.751643
Epoch 221
Validation auc = 0.765023
Epoch 222
Validation auc = 0.767606
Epoch 223
Validation auc = 0.750469
Epoch 224
Loss = 4.5656e+00, PNorm = 92.2654, GNorm = 246.7326, lr_0 = 6.8953e-04
Validation auc = 0.746479
Epoch 225
Validation auc = 0.721127
Epoch 226
Validation auc = 0.690845
Epoch 227
Validation auc = 0.680516
Epoch 228
Validation auc = 0.667606
Epoch 229
Loss = 2.6802e+00, PNorm = 92.5542, GNorm = 178.1346, lr_0 = 6.8364e-04
Validation auc = 0.658920
Epoch 230
Validation auc = 0.654460
Epoch 231
Validation auc = 0.656808
Epoch 232
Validation auc = 0.671831
Epoch 233
Validation auc = 0.688028
Epoch 234
Loss = 1.2676e+00, PNorm = 92.8288, GNorm = 79.7192, lr_0 = 6.7779e-04
Validation auc = 0.709155
Epoch 235
Validation auc = 0.720657
Epoch 236
Validation auc = 0.726526
Epoch 237
Validation auc = 0.738498
Epoch 238
Validation auc = 0.747887
Epoch 239
Loss = 2.7702e+00, PNorm = 93.0771, GNorm = 50.2926, lr_0 = 6.7200e-04
Validation auc = 0.755164
Epoch 240
Validation auc = 0.749061
Epoch 241
Validation auc = 0.742254
Epoch 242
Validation auc = 0.740845
Epoch 243
Validation auc = 0.735681
Epoch 244
Loss = 6.6018e-01, PNorm = 93.2674, GNorm = 66.2615, lr_0 = 6.6625e-04
Validation auc = 0.736150
Epoch 245
Validation auc = 0.735681
Epoch 246
Validation auc = 0.729108
Epoch 247
Validation auc = 0.726995
Epoch 248
Validation auc = 0.714554
Epoch 249
Loss = 1.8912e+00, PNorm = 93.4703, GNorm = 181.2808, lr_0 = 6.6056e-04
Validation auc = 0.706338
Epoch 250
Validation auc = 0.695540
Epoch 251
Validation auc = 0.688732
Epoch 252
Validation auc = 0.686854
Epoch 253
Validation auc = 0.692488
Epoch 254
Loss = 2.7391e+00, PNorm = 93.7207, GNorm = 195.4281, lr_0 = 6.5491e-04
Validation auc = 0.700469
Epoch 255
Validation auc = 0.706808
Epoch 256
Validation auc = 0.704460
Epoch 257
Validation auc = 0.706808
Epoch 258
Validation auc = 0.719014
Epoch 259
Loss = 1.4831e+00, PNorm = 93.9037, GNorm = 95.8525, lr_0 = 6.4931e-04
Validation auc = 0.730047
Epoch 260
Validation auc = 0.733333
Epoch 261
Validation auc = 0.734742
Epoch 262
Validation auc = 0.738028
Epoch 263
Validation auc = 0.737559
Epoch 264
Loss = 1.7288e+00, PNorm = 94.0723, GNorm = 191.1672, lr_0 = 6.4376e-04
Validation auc = 0.732864
Epoch 265
Validation auc = 0.708920
Epoch 266
Validation auc = 0.678873
Epoch 267
Validation auc = 0.673005
Epoch 268
Validation auc = 0.692958
Epoch 269
Loss = 1.1467e+00, PNorm = 94.3243, GNorm = 186.1312, lr_0 = 6.3826e-04
Validation auc = 0.698357
Epoch 270
Validation auc = 0.707277
Epoch 271
Validation auc = 0.710798
Epoch 272
Validation auc = 0.711737
Epoch 273
Validation auc = 0.716197
Epoch 274
Loss = 8.7224e-01, PNorm = 94.5416, GNorm = 132.2240, lr_0 = 6.3280e-04
Validation auc = 0.724413
Epoch 275
Validation auc = 0.731455
Epoch 276
Validation auc = 0.730516
Epoch 277
Validation auc = 0.721127
Epoch 278
Validation auc = 0.726995
Epoch 279
Loss = 1.7063e+00, PNorm = 94.7710, GNorm = 127.7915, lr_0 = 6.2739e-04
Validation auc = 0.730751
Epoch 280
Validation auc = 0.729812
Epoch 281
Validation auc = 0.732394
Epoch 282
Validation auc = 0.738498
Epoch 283
Validation auc = 0.732394
Epoch 284
Loss = 4.3207e-01, PNorm = 95.0072, GNorm = 72.2645, lr_0 = 6.2203e-04
Validation auc = 0.734272
Epoch 285
Validation auc = 0.731925
Epoch 286
Validation auc = 0.737089
Epoch 287
Validation auc = 0.744366
Epoch 288
Validation auc = 0.732864
Epoch 289
Loss = 1.5765e+00, PNorm = 95.2111, GNorm = 124.1895, lr_0 = 6.1671e-04
Validation auc = 0.728169
Epoch 290
Validation auc = 0.723944
Epoch 291
Validation auc = 0.718310
Epoch 292
Validation auc = 0.718075
Epoch 293
Validation auc = 0.713850
Epoch 294
Loss = 1.7485e+00, PNorm = 95.3727, GNorm = 151.5102, lr_0 = 6.1144e-04
Validation auc = 0.713146
Epoch 295
Validation auc = 0.717371
Epoch 296
Validation auc = 0.717371
Epoch 297
Validation auc = 0.714085
Epoch 298
Validation auc = 0.703521
Epoch 299
Loss = 7.8911e-01, PNorm = 95.5096, GNorm = 171.9025, lr_0 = 6.0621e-04
Validation auc = 0.696479
Model 0 best validation auc = 0.779343 on epoch 125
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.764299
Ensemble test auc = 0.764299
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.669484
Epoch 1
Validation auc = 0.711033
Epoch 2
Validation auc = 0.685681
Epoch 3
Validation auc = 0.738263
Epoch 4
Loss = 3.6308e+01, PNorm = 78.7083, GNorm = 869.3078, lr_0 = 6.5000e-04
Validation auc = 0.702817
Epoch 5
Validation auc = 0.665258
Epoch 6
Validation auc = 0.729577
Epoch 7
Validation auc = 0.727465
Epoch 8
Validation auc = 0.704225
Epoch 9
Loss = 4.9920e+01, PNorm = 79.0272, GNorm = 798.3774, lr_0 = 9.9743e-04
Validation auc = 0.711972
Epoch 10
Validation auc = 0.712676
Epoch 11
Validation auc = 0.680516
Epoch 12
Validation auc = 0.657277
Epoch 13
Validation auc = 0.704930
Epoch 14
Loss = 4.5304e+01, PNorm = 79.3674, GNorm = 680.7617, lr_0 = 9.8890e-04
Validation auc = 0.725822
Epoch 15
Validation auc = 0.658216
Epoch 16
Validation auc = 0.672535
Epoch 17
Validation auc = 0.697418
Epoch 18
Validation auc = 0.688263
Epoch 19
Loss = 4.0628e+01, PNorm = 79.6306, GNorm = 968.6025, lr_0 = 9.8045e-04
Validation auc = 0.603521
Epoch 20
Validation auc = 0.675117
Epoch 21
Validation auc = 0.738498
Epoch 22
Validation auc = 0.719953
Epoch 23
Validation auc = 0.643662
Epoch 24
Loss = 6.2993e+01, PNorm = 79.8111, GNorm = 1747.2338, lr_0 = 9.7207e-04
Validation auc = 0.736620
Epoch 25
Validation auc = 0.684038
Epoch 26
Validation auc = 0.719484
Epoch 27
Validation auc = 0.685681
Epoch 28
Validation auc = 0.738498
Epoch 29
Loss = 2.2500e+01, PNorm = 79.9543, GNorm = 677.6791, lr_0 = 9.6376e-04
Validation auc = 0.759624
Epoch 30
Validation auc = 0.716432
Epoch 31
Validation auc = 0.674648
Epoch 32
Validation auc = 0.731925
Epoch 33
Validation auc = 0.738498
Epoch 34
Loss = 2.3456e+01, PNorm = 80.1101, GNorm = 317.9592, lr_0 = 9.5552e-04
Validation auc = 0.685446
Epoch 35
Validation auc = 0.691549
Epoch 36
Validation auc = 0.723005
Epoch 37
Validation auc = 0.716432
Epoch 38
Validation auc = 0.686854
Epoch 39
Loss = 3.5356e+01, PNorm = 80.2914, GNorm = 1053.6927, lr_0 = 9.4735e-04
Validation auc = 0.683333
Epoch 40
Validation auc = 0.753991
Epoch 41
Validation auc = 0.753286
Epoch 42
Validation auc = 0.746948
Epoch 43
Validation auc = 0.707512
Epoch 44
Loss = 2.8795e+01, PNorm = 80.4298, GNorm = 518.7100, lr_0 = 9.3925e-04
Validation auc = 0.747418
Epoch 45
Validation auc = 0.720657
Epoch 46
Validation auc = 0.680282
Epoch 47
Validation auc = 0.707512
Epoch 48
Validation auc = 0.734038
Epoch 49
Loss = 2.0892e+01, PNorm = 80.5945, GNorm = 649.7411, lr_0 = 9.3122e-04
Validation auc = 0.741784
Epoch 50
Validation auc = 0.707042
Epoch 51
Validation auc = 0.651878
Epoch 52
Validation auc = 0.693897
Epoch 53
Validation auc = 0.695540
Epoch 54
Loss = 3.3519e+01, PNorm = 80.8206, GNorm = 1324.1930, lr_0 = 9.2326e-04
Validation auc = 0.702582
Epoch 55
Validation auc = 0.608216
Epoch 56
Validation auc = 0.632160
Epoch 57
Validation auc = 0.719249
Epoch 58
Validation auc = 0.704695
Epoch 59
Loss = 1.2007e+01, PNorm = 81.0446, GNorm = 346.7082, lr_0 = 9.1537e-04
Validation auc = 0.671362
Epoch 60
Validation auc = 0.674883
Epoch 61
Validation auc = 0.700704
Epoch 62
Validation auc = 0.740141
Epoch 63
Validation auc = 0.770423
Epoch 64
Loss = 1.6809e+01, PNorm = 81.4935, GNorm = 483.1561, lr_0 = 9.0754e-04
Validation auc = 0.736854
Epoch 65
Validation auc = 0.718545
Epoch 66
Validation auc = 0.735211
Epoch 67
Validation auc = 0.734038
Epoch 68
Validation auc = 0.755164
Epoch 69
Loss = 1.1035e+01, PNorm = 81.7421, GNorm = 543.3388, lr_0 = 8.9978e-04
Validation auc = 0.674413
Epoch 70
Validation auc = 0.717840
Epoch 71
Validation auc = 0.758451
Epoch 72
Validation auc = 0.750000
Epoch 73
Validation auc = 0.717840
Epoch 74
Loss = 1.8770e+01, PNorm = 82.0322, GNorm = 332.3654, lr_0 = 8.9209e-04
Validation auc = 0.739202
Epoch 75
Validation auc = 0.749296
Epoch 76
Validation auc = 0.767371
Epoch 77
Validation auc = 0.758685
Epoch 78
Validation auc = 0.751408
Epoch 79
Loss = 1.3166e+01, PNorm = 82.3847, GNorm = 535.3461, lr_0 = 8.8447e-04
Validation auc = 0.747418
Epoch 80
Validation auc = 0.761502
Epoch 81
Validation auc = 0.713146
Epoch 82
Validation auc = 0.729108
Epoch 83
Validation auc = 0.741784
Epoch 84
Loss = 1.0239e+01, PNorm = 82.7092, GNorm = 303.5390, lr_0 = 8.7691e-04
Validation auc = 0.716197
Epoch 85
Validation auc = 0.735681
Epoch 86
Validation auc = 0.730751
Epoch 87
Validation auc = 0.665728
Epoch 88
Validation auc = 0.663146
Epoch 89
Loss = 1.8072e+01, PNorm = 83.1113, GNorm = 342.4873, lr_0 = 8.6941e-04
Validation auc = 0.734507
Epoch 90
Validation auc = 0.734272
Epoch 91
Validation auc = 0.671596
Epoch 92
Validation auc = 0.650939
Epoch 93
Validation auc = 0.696948
Epoch 94
Loss = 1.1042e+01, PNorm = 83.5602, GNorm = 452.3871, lr_0 = 8.6198e-04
Validation auc = 0.724178
Epoch 95
Validation auc = 0.720892
Epoch 96
Validation auc = 0.724648
Epoch 97
Validation auc = 0.731925
Epoch 98
Validation auc = 0.717371
Epoch 99
Loss = 6.3521e+00, PNorm = 84.1260, GNorm = 537.5653, lr_0 = 8.5461e-04
Validation auc = 0.729343
Epoch 100
Validation auc = 0.710094
Epoch 101
Validation auc = 0.746714
Epoch 102
Validation auc = 0.766432
Epoch 103
Validation auc = 0.769484
Epoch 104
Loss = 9.1555e+00, PNorm = 84.5664, GNorm = 232.4712, lr_0 = 8.4730e-04
Validation auc = 0.757042
Epoch 105
Validation auc = 0.728638
Epoch 106
Validation auc = 0.709624
Epoch 107
Validation auc = 0.713380
Epoch 108
Validation auc = 0.707042
Epoch 109
Loss = 7.5315e+00, PNorm = 85.1108, GNorm = 369.3106, lr_0 = 8.4006e-04
Validation auc = 0.730751
Epoch 110
Validation auc = 0.760563
Epoch 111
Validation auc = 0.766901
Epoch 112
Validation auc = 0.770423
Epoch 113
Validation auc = 0.756338
Epoch 114
Loss = 6.1654e+00, PNorm = 85.6170, GNorm = 363.7232, lr_0 = 8.3288e-04
Validation auc = 0.746009
Epoch 115
Validation auc = 0.758685
Epoch 116
Validation auc = 0.711737
Epoch 117
Validation auc = 0.730047
Epoch 118
Validation auc = 0.747653
Epoch 119
Loss = 8.6701e+00, PNorm = 86.1889, GNorm = 367.5733, lr_0 = 8.2576e-04
Validation auc = 0.740376
Epoch 120
Validation auc = 0.690845
Epoch 121
Validation auc = 0.681690
Epoch 122
Validation auc = 0.747183
Epoch 123
Validation auc = 0.752582
Epoch 124
Loss = 6.7749e+00, PNorm = 86.8912, GNorm = 205.3685, lr_0 = 8.1870e-04
Validation auc = 0.719249
Epoch 125
Validation auc = 0.686150
Epoch 126
Validation auc = 0.719484
Epoch 127
Validation auc = 0.727465
Epoch 128
Validation auc = 0.700235
Epoch 129
Loss = 5.3264e+00, PNorm = 87.3814, GNorm = 289.8613, lr_0 = 8.1170e-04
Validation auc = 0.681690
Epoch 130
Validation auc = 0.681221
Epoch 131
Validation auc = 0.721596
Epoch 132
Validation auc = 0.719014
Epoch 133
Validation auc = 0.719718
Epoch 134
Loss = 5.8484e+00, PNorm = 87.9118, GNorm = 791.1121, lr_0 = 8.0476e-04
Validation auc = 0.725587
Epoch 135
Validation auc = 0.731925
Epoch 136
Validation auc = 0.722300
Epoch 137
Validation auc = 0.706573
Epoch 138
Validation auc = 0.686620
Epoch 139
Loss = 5.1325e+00, PNorm = 88.2561, GNorm = 189.9447, lr_0 = 7.9788e-04
Validation auc = 0.715023
Epoch 140
Validation auc = 0.727934
Epoch 141
Validation auc = 0.727700
Epoch 142
Validation auc = 0.713146
Epoch 143
Validation auc = 0.708451
Epoch 144
Loss = 3.1599e+00, PNorm = 88.7267, GNorm = 185.8258, lr_0 = 7.9106e-04
Validation auc = 0.725587
Epoch 145
Validation auc = 0.735446
Epoch 146
Validation auc = 0.724648
Epoch 147
Validation auc = 0.711502
Epoch 148
Validation auc = 0.738263
Epoch 149
Loss = 4.5934e+00, PNorm = 89.0728, GNorm = 227.8805, lr_0 = 7.8430e-04
Validation auc = 0.750704
Epoch 150
Validation auc = 0.748592
Epoch 151
Validation auc = 0.733803
Epoch 152
Validation auc = 0.736385
Epoch 153
Validation auc = 0.743662
Epoch 154
Loss = 7.7736e+00, PNorm = 89.4407, GNorm = 436.2141, lr_0 = 7.7759e-04
Validation auc = 0.734507
Epoch 155
Validation auc = 0.719014
Epoch 156
Validation auc = 0.737089
Epoch 157
Validation auc = 0.764319
Epoch 158
Validation auc = 0.783333
Epoch 159
Loss = 7.2335e+00, PNorm = 89.8972, GNorm = 560.8201, lr_0 = 7.7095e-04
Validation auc = 0.765258
Epoch 160
Validation auc = 0.719484
Epoch 161
Validation auc = 0.753052
Epoch 162
Validation auc = 0.790610
Epoch 163
Validation auc = 0.799061
Epoch 164
Loss = 7.3651e+00, PNorm = 90.3517, GNorm = 506.1306, lr_0 = 7.6436e-04
Validation auc = 0.781925
Epoch 165
Validation auc = 0.742723
Epoch 166
Validation auc = 0.725352
Epoch 167
Validation auc = 0.745070
Epoch 168
Validation auc = 0.776056
Epoch 169
Loss = 2.5750e+00, PNorm = 90.6988, GNorm = 336.0613, lr_0 = 7.5782e-04
Validation auc = 0.781221
Epoch 170
Validation auc = 0.781690
Epoch 171
Validation auc = 0.767371
Epoch 172
Validation auc = 0.747418
Epoch 173
Validation auc = 0.718779
Epoch 174
Loss = 8.8260e+00, PNorm = 91.0318, GNorm = 357.3321, lr_0 = 7.5134e-04
Validation auc = 0.758216
Epoch 175
Validation auc = 0.784272
Epoch 176
Validation auc = 0.798592
Epoch 177
Validation auc = 0.784742
Epoch 178
Validation auc = 0.761502
Epoch 179
Loss = 6.5862e+00, PNorm = 91.4512, GNorm = 304.8615, lr_0 = 7.4492e-04
Validation auc = 0.737089
Epoch 180
Validation auc = 0.750235
Epoch 181
Validation auc = 0.755634
Epoch 182
Validation auc = 0.751408
Epoch 183
Validation auc = 0.749296
Epoch 184
Loss = 3.1156e+00, PNorm = 91.7246, GNorm = 229.7577, lr_0 = 7.3855e-04
Validation auc = 0.727465
Epoch 185
Validation auc = 0.698826
Epoch 186
Validation auc = 0.687089
Epoch 187
Validation auc = 0.704460
Epoch 188
Validation auc = 0.734038
Epoch 189
Loss = 2.1008e+00, PNorm = 92.0625, GNorm = 117.8449, lr_0 = 7.3224e-04
Validation auc = 0.751408
Epoch 190
Validation auc = 0.758216
Epoch 191
Validation auc = 0.749061
Epoch 192
Validation auc = 0.734977
Epoch 193
Validation auc = 0.731925
Epoch 194
Loss = 3.9972e+00, PNorm = 92.4265, GNorm = 58.5009, lr_0 = 7.2598e-04
Validation auc = 0.746009
Epoch 195
Validation auc = 0.762207
Epoch 196
Validation auc = 0.761737
Epoch 197
Validation auc = 0.753756
Epoch 198
Validation auc = 0.752817
Epoch 199
Loss = 1.1165e+00, PNorm = 92.7382, GNorm = 177.1023, lr_0 = 7.1977e-04
Validation auc = 0.748592
Epoch 200
Validation auc = 0.744366
Epoch 201
Validation auc = 0.725822
Epoch 202
Validation auc = 0.713380
Epoch 203
Validation auc = 0.716197
Epoch 204
Loss = 1.3744e+00, PNorm = 93.0529, GNorm = 146.1674, lr_0 = 7.1362e-04
Validation auc = 0.729108
Epoch 205
Validation auc = 0.723709
Epoch 206
Validation auc = 0.704460
Epoch 207
Validation auc = 0.693897
Epoch 208
Validation auc = 0.707277
Epoch 209
Loss = 3.8750e+00, PNorm = 93.3638, GNorm = 231.2132, lr_0 = 7.0752e-04
Validation auc = 0.719953
Epoch 210
Validation auc = 0.722300
Epoch 211
Validation auc = 0.727465
Epoch 212
Validation auc = 0.728169
Epoch 213
Validation auc = 0.732864
Epoch 214
Loss = 9.2115e-01, PNorm = 93.6472, GNorm = 123.7673, lr_0 = 7.0147e-04
Validation auc = 0.734507
Epoch 215
Validation auc = 0.745775
Epoch 216
Validation auc = 0.751878
Epoch 217
Validation auc = 0.748357
Epoch 218
Validation auc = 0.754695
Epoch 219
Loss = 1.0969e+00, PNorm = 93.9033, GNorm = 137.8599, lr_0 = 6.9548e-04
Validation auc = 0.755869
Epoch 220
Validation auc = 0.749061
Epoch 221
Validation auc = 0.736854
Epoch 222
Validation auc = 0.725587
Epoch 223
Validation auc = 0.720657
Epoch 224
Loss = 2.2643e+00, PNorm = 94.2113, GNorm = 80.9320, lr_0 = 6.8953e-04
Validation auc = 0.718310
Epoch 225
Validation auc = 0.694836
Epoch 226
Validation auc = 0.696948
Epoch 227
Validation auc = 0.697887
Epoch 228
Validation auc = 0.700704
Epoch 229
Loss = 2.1945e+00, PNorm = 94.4469, GNorm = 175.2856, lr_0 = 6.8364e-04
Validation auc = 0.717606
Epoch 230
Validation auc = 0.729343
Epoch 231
Validation auc = 0.726056
Epoch 232
Validation auc = 0.735915
Epoch 233
Validation auc = 0.744836
Epoch 234
Loss = 2.5780e+00, PNorm = 94.7088, GNorm = 89.0007, lr_0 = 6.7779e-04
Validation auc = 0.774883
Epoch 235
Validation auc = 0.780986
Epoch 236
Validation auc = 0.769484
Epoch 237
Validation auc = 0.769249
Epoch 238
Validation auc = 0.762676
Epoch 239
Loss = 3.4351e+00, PNorm = 95.0588, GNorm = 163.9488, lr_0 = 6.7200e-04
Validation auc = 0.760094
Epoch 240
Validation auc = 0.761502
Epoch 241
Validation auc = 0.762911
Epoch 242
Validation auc = 0.775587
Epoch 243
Validation auc = 0.784038
Epoch 244
Loss = 2.0233e+00, PNorm = 95.4264, GNorm = 124.9408, lr_0 = 6.6625e-04
Validation auc = 0.788498
Epoch 245
Validation auc = 0.782160
Epoch 246
Validation auc = 0.765962
Epoch 247
Validation auc = 0.742019
Epoch 248
Validation auc = 0.740610
Epoch 249
Loss = 8.8904e-01, PNorm = 95.7430, GNorm = 83.5107, lr_0 = 6.6056e-04
Validation auc = 0.743427
Epoch 250
Validation auc = 0.759624
Epoch 251
Validation auc = 0.765023
Epoch 252
Validation auc = 0.768779
Epoch 253
Validation auc = 0.769953
Epoch 254
Loss = 3.0248e+00, PNorm = 96.0200, GNorm = 232.3592, lr_0 = 6.5491e-04
Validation auc = 0.766432
Epoch 255
Validation auc = 0.763850
Epoch 256
Validation auc = 0.757042
Epoch 257
Validation auc = 0.761972
Epoch 258
Validation auc = 0.767136
Epoch 259
Loss = 2.1826e+00, PNorm = 96.2846, GNorm = 132.4356, lr_0 = 6.4931e-04
Validation auc = 0.766667
Epoch 260
Validation auc = 0.762911
Epoch 261
Validation auc = 0.761502
Epoch 262
Validation auc = 0.748592
Epoch 263
Validation auc = 0.738732
Epoch 264
Loss = 1.2167e+00, PNorm = 96.5395, GNorm = 60.4206, lr_0 = 6.4376e-04
Validation auc = 0.750000
Epoch 265
Validation auc = 0.757042
Epoch 266
Validation auc = 0.756573
Epoch 267
Validation auc = 0.753286
Epoch 268
Validation auc = 0.750000
Epoch 269
Loss = 8.7002e-01, PNorm = 96.7980, GNorm = 118.7168, lr_0 = 6.3826e-04
Validation auc = 0.755869
Epoch 270
Validation auc = 0.763380
Epoch 271
Validation auc = 0.750704
Epoch 272
Validation auc = 0.739202
Epoch 273
Validation auc = 0.737089
Epoch 274
Loss = 1.2158e+00, PNorm = 97.0346, GNorm = 173.0710, lr_0 = 6.3280e-04
Validation auc = 0.750235
Epoch 275
Validation auc = 0.756573
Epoch 276
Validation auc = 0.752113
Epoch 277
Validation auc = 0.747418
Epoch 278
Validation auc = 0.750939
Epoch 279
Loss = 1.2563e+00, PNorm = 97.2737, GNorm = 76.7256, lr_0 = 6.2739e-04
Validation auc = 0.749531
Epoch 280
Validation auc = 0.742723
Epoch 281
Validation auc = 0.746244
Epoch 282
Validation auc = 0.750469
Epoch 283
Validation auc = 0.746009
Epoch 284
Loss = 7.2010e-01, PNorm = 97.5184, GNorm = 149.3482, lr_0 = 6.2203e-04
Validation auc = 0.738028
Epoch 285
Validation auc = 0.739906
Epoch 286
Validation auc = 0.734742
Epoch 287
Validation auc = 0.736854
Epoch 288
Validation auc = 0.752582
Epoch 289
Loss = 7.6461e-01, PNorm = 97.7946, GNorm = 80.5548, lr_0 = 6.1671e-04
Validation auc = 0.753756
Epoch 290
Validation auc = 0.761502
Epoch 291
Validation auc = 0.769014
Epoch 292
Validation auc = 0.777465
Epoch 293
Validation auc = 0.781925
Epoch 294
Loss = 9.1885e-01, PNorm = 98.0845, GNorm = 76.3703, lr_0 = 6.1144e-04
Validation auc = 0.777934
Epoch 295
Validation auc = 0.767840
Epoch 296
Validation auc = 0.765023
Epoch 297
Validation auc = 0.759859
Epoch 298
Validation auc = 0.749531
Epoch 299
Loss = 9.0351e-01, PNorm = 98.2852, GNorm = 153.7967, lr_0 = 6.0621e-04
Validation auc = 0.742254
Model 0 best validation auc = 0.799061 on epoch 163
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.772426
Ensemble test auc = 0.772426
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.514085
Epoch 1
Validation auc = 0.536620
Epoch 2
Validation auc = 0.630986
Epoch 3
Validation auc = 0.732864
Epoch 4
Loss = 6.4170e+01, PNorm = 78.6732, GNorm = 1902.7717, lr_0 = 6.5000e-04
Validation auc = 0.741080
Epoch 5
Validation auc = 0.679577
Epoch 6
Validation auc = 0.693427
Epoch 7
Validation auc = 0.703756
Epoch 8
Validation auc = 0.746948
Epoch 9
Loss = 4.2646e+01, PNorm = 78.9365, GNorm = 1156.5274, lr_0 = 9.9743e-04
Validation auc = 0.766667
Epoch 10
Validation auc = 0.749061
Epoch 11
Validation auc = 0.742488
Epoch 12
Validation auc = 0.726056
Epoch 13
Validation auc = 0.735681
Epoch 14
Loss = 2.5674e+01, PNorm = 79.2007, GNorm = 643.6729, lr_0 = 9.8890e-04
Validation auc = 0.735211
Epoch 15
Validation auc = 0.686620
Epoch 16
Validation auc = 0.688263
Epoch 17
Validation auc = 0.704460
Epoch 18
Validation auc = 0.696948
Epoch 19
Loss = 3.5980e+01, PNorm = 79.3775, GNorm = 764.1974, lr_0 = 9.8045e-04
Validation auc = 0.669953
Epoch 20
Validation auc = 0.726291
Epoch 21
Validation auc = 0.695540
Epoch 22
Validation auc = 0.714085
Epoch 23
Validation auc = 0.689202
Epoch 24
Loss = 4.0237e+01, PNorm = 79.5123, GNorm = 670.9793, lr_0 = 9.7207e-04
Validation auc = 0.710563
Epoch 25
Validation auc = 0.696714
Epoch 26
Validation auc = 0.699061
Epoch 27
Validation auc = 0.695775
Epoch 28
Validation auc = 0.686150
Epoch 29
Loss = 5.2327e+01, PNorm = 79.6467, GNorm = 1042.4264, lr_0 = 9.6376e-04
Validation auc = 0.692488
Epoch 30
Validation auc = 0.668075
Epoch 31
Validation auc = 0.706808
Epoch 32
Validation auc = 0.658685
Epoch 33
Validation auc = 0.650704
Epoch 34
Loss = 4.2951e+01, PNorm = 79.8091, GNorm = 937.3775, lr_0 = 9.5552e-04
Validation auc = 0.644366
Epoch 35
Validation auc = 0.689202
Epoch 36
Validation auc = 0.708685
Epoch 37
Validation auc = 0.691784
Epoch 38
Validation auc = 0.662911
Epoch 39
Loss = 5.5756e+01, PNorm = 79.9859, GNorm = 1241.1168, lr_0 = 9.4735e-04
Validation auc = 0.757746
Epoch 40
Validation auc = 0.714085
Epoch 41
Validation auc = 0.711737
Epoch 42
Validation auc = 0.738028
Epoch 43
Validation auc = 0.743662
Epoch 44
Loss = 2.7262e+01, PNorm = 80.2254, GNorm = 624.9511, lr_0 = 9.3925e-04
Validation auc = 0.747418
Epoch 45
Validation auc = 0.698592
Epoch 46
Validation auc = 0.722535
Epoch 47
Validation auc = 0.724413
Epoch 48
Validation auc = 0.652113
Epoch 49
Loss = 3.2258e+01, PNorm = 80.6922, GNorm = 1687.7035, lr_0 = 9.3122e-04
Validation auc = 0.683099
Epoch 50
Validation auc = 0.664085
Epoch 51
Validation auc = 0.708920
Epoch 52
Validation auc = 0.724648
Epoch 53
Validation auc = 0.735915
Epoch 54
Loss = 1.2903e+01, PNorm = 80.9845, GNorm = 702.8568, lr_0 = 9.2326e-04
Validation auc = 0.689202
Epoch 55
Validation auc = 0.714554
Epoch 56
Validation auc = 0.681690
Epoch 57
Validation auc = 0.641549
Epoch 58
Validation auc = 0.731925
Epoch 59
Loss = 2.4068e+01, PNorm = 81.4500, GNorm = 394.2065, lr_0 = 9.1537e-04
Validation auc = 0.736150
Epoch 60
Validation auc = 0.689906
Epoch 61
Validation auc = 0.646009
Epoch 62
Validation auc = 0.669484
Epoch 63
Validation auc = 0.708920
Epoch 64
Loss = 1.4380e+01, PNorm = 81.8610, GNorm = 303.3995, lr_0 = 9.0754e-04
Validation auc = 0.683568
Epoch 65
Validation auc = 0.675117
Epoch 66
Validation auc = 0.683568
Epoch 67
Validation auc = 0.699296
Epoch 68
Validation auc = 0.666901
Epoch 69
Loss = 1.7173e+01, PNorm = 82.1728, GNorm = 906.1722, lr_0 = 8.9978e-04
Validation auc = 0.612207
Epoch 70
Validation auc = 0.693662
Epoch 71
Validation auc = 0.740845
Epoch 72
Validation auc = 0.713380
Epoch 73
Validation auc = 0.610329
Epoch 74
Loss = 2.1699e+01, PNorm = 82.4226, GNorm = 1062.4836, lr_0 = 8.9209e-04
Validation auc = 0.604695
Epoch 75
Validation auc = 0.685446
Epoch 76
Validation auc = 0.750235
Epoch 77
Validation auc = 0.738498
Epoch 78
Validation auc = 0.671831
Epoch 79
Loss = 2.4471e+01, PNorm = 82.8017, GNorm = 709.0680, lr_0 = 8.8447e-04
Validation auc = 0.622535
Epoch 80
Validation auc = 0.680751
Epoch 81
Validation auc = 0.709155
Epoch 82
Validation auc = 0.704930
Epoch 83
Validation auc = 0.662911
Epoch 84
Loss = 2.6335e+01, PNorm = 83.1228, GNorm = 703.2042, lr_0 = 8.7691e-04
Validation auc = 0.657512
Epoch 85
Validation auc = 0.693427
Epoch 86
Validation auc = 0.721362
Epoch 87
Validation auc = 0.730047
Epoch 88
Validation auc = 0.726995
Epoch 89
Loss = 1.5173e+01, PNorm = 83.5284, GNorm = 723.2493, lr_0 = 8.6941e-04
Validation auc = 0.724883
Epoch 90
Validation auc = 0.737324
Epoch 91
Validation auc = 0.720892
Epoch 92
Validation auc = 0.735446
Epoch 93
Validation auc = 0.778873
Epoch 94
Loss = 8.0092e+00, PNorm = 83.9829, GNorm = 390.6529, lr_0 = 8.6198e-04
Validation auc = 0.792019
Epoch 95
Validation auc = 0.787324
Epoch 96
Validation auc = 0.745305
Epoch 97
Validation auc = 0.733333
Epoch 98
Validation auc = 0.758216
Epoch 99
Loss = 1.2655e+01, PNorm = 84.4058, GNorm = 500.4996, lr_0 = 8.5461e-04
Validation auc = 0.770892
Epoch 100
Validation auc = 0.756808
Epoch 101
Validation auc = 0.743897
Epoch 102
Validation auc = 0.719718
Epoch 103
Validation auc = 0.705164
Epoch 104
Loss = 8.4869e+00, PNorm = 84.8073, GNorm = 229.4356, lr_0 = 8.4730e-04
Validation auc = 0.716432
Epoch 105
Validation auc = 0.742488
Epoch 106
Validation auc = 0.739906
Epoch 107
Validation auc = 0.715023
Epoch 108
Validation auc = 0.695305
Epoch 109
Loss = 1.8004e+01, PNorm = 85.1352, GNorm = 1003.8278, lr_0 = 8.4006e-04
Validation auc = 0.725822
Epoch 110
Validation auc = 0.772300
Epoch 111
Validation auc = 0.783099
Epoch 112
Validation auc = 0.748357
Epoch 113
Validation auc = 0.655164
Epoch 114
Loss = 1.2283e+01, PNorm = 85.5031, GNorm = 384.4777, lr_0 = 8.3288e-04
Validation auc = 0.652817
Epoch 115
Validation auc = 0.704460
Epoch 116
Validation auc = 0.761737
Epoch 117
Validation auc = 0.763380
Epoch 118
Validation auc = 0.742019
Epoch 119
Loss = 6.0770e+00, PNorm = 85.9821, GNorm = 261.6751, lr_0 = 8.2576e-04
Validation auc = 0.715962
Epoch 120
Validation auc = 0.690610
Epoch 121
Validation auc = 0.689437
Epoch 122
Validation auc = 0.729108
Epoch 123
Validation auc = 0.737089
Epoch 124
Loss = 7.2893e+00, PNorm = 86.3187, GNorm = 268.1937, lr_0 = 8.1870e-04
Validation auc = 0.727465
Epoch 125
Validation auc = 0.735211
Epoch 126
Validation auc = 0.692958
Epoch 127
Validation auc = 0.666667
Epoch 128
Validation auc = 0.680516
Epoch 129
Loss = 9.8391e+00, PNorm = 86.5750, GNorm = 420.0906, lr_0 = 8.1170e-04
Validation auc = 0.727230
Epoch 130
Validation auc = 0.700704
Epoch 131
Validation auc = 0.737793
Epoch 132
Validation auc = 0.692723
Epoch 133
Validation auc = 0.669718
Epoch 134
Loss = 7.8639e+00, PNorm = 86.8967, GNorm = 338.9754, lr_0 = 8.0476e-04
Validation auc = 0.675352
Epoch 135
Validation auc = 0.701174
Epoch 136
Validation auc = 0.726291
Epoch 137
Validation auc = 0.728169
Epoch 138
Validation auc = 0.715962
Epoch 139
Loss = 7.1014e+00, PNorm = 87.1540, GNorm = 250.8298, lr_0 = 7.9788e-04
Validation auc = 0.701174
Epoch 140
Validation auc = 0.700704
Epoch 141
Validation auc = 0.694366
Epoch 142
Validation auc = 0.694601
Epoch 143
Validation auc = 0.702817
Epoch 144
Loss = 9.1789e+00, PNorm = 87.4909, GNorm = 281.1387, lr_0 = 7.9106e-04
Validation auc = 0.676995
Epoch 145
Validation auc = 0.625352
Epoch 146
Validation auc = 0.604930
Epoch 147
Validation auc = 0.636150
Epoch 148
Validation auc = 0.662911
Epoch 149
Loss = 4.2394e+00, PNorm = 87.8308, GNorm = 89.8504, lr_0 = 7.8430e-04
Validation auc = 0.687089
Epoch 150
Validation auc = 0.718779
Epoch 151
Validation auc = 0.721831
Epoch 152
Validation auc = 0.735211
Epoch 153
Validation auc = 0.738498
Epoch 154
Loss = 4.8647e+00, PNorm = 88.1194, GNorm = 254.3592, lr_0 = 7.7759e-04
Validation auc = 0.737089
Epoch 155
Validation auc = 0.734742
Epoch 156
Validation auc = 0.739906
Epoch 157
Validation auc = 0.723239
Epoch 158
Validation auc = 0.738967
Epoch 159
Loss = 6.9497e+00, PNorm = 88.5205, GNorm = 329.9865, lr_0 = 7.7095e-04
Validation auc = 0.757746
Epoch 160
Validation auc = 0.763146
Epoch 161
Validation auc = 0.778169
Epoch 162
Validation auc = 0.769484
Epoch 163
Validation auc = 0.763146
Epoch 164
Loss = 3.8590e+00, PNorm = 88.9352, GNorm = 284.7654, lr_0 = 7.6436e-04
Validation auc = 0.774178
Epoch 165
Validation auc = 0.779812
Epoch 166
Validation auc = 0.765728
Epoch 167
Validation auc = 0.761737
Epoch 168
Validation auc = 0.760798
Epoch 169
Loss = 3.5268e+00, PNorm = 89.3442, GNorm = 257.0560, lr_0 = 7.5782e-04
Validation auc = 0.764319
Epoch 170
Validation auc = 0.767371
Epoch 171
Validation auc = 0.775822
Epoch 172
Validation auc = 0.751408
Epoch 173
Validation auc = 0.737089
Epoch 174
Loss = 4.3801e+00, PNorm = 89.6523, GNorm = 139.0250, lr_0 = 7.5134e-04
Validation auc = 0.769249
Epoch 175
Validation auc = 0.796479
Epoch 176
Validation auc = 0.794601
Epoch 177
Validation auc = 0.788732
Epoch 178
Validation auc = 0.796009
Epoch 179
Loss = 4.3827e+00, PNorm = 90.0394, GNorm = 437.0826, lr_0 = 7.4492e-04
Validation auc = 0.801174
Epoch 180
Validation auc = 0.783568
Epoch 181
Validation auc = 0.772770
Epoch 182
Validation auc = 0.776056
Epoch 183
Validation auc = 0.777230
Epoch 184
Loss = 7.5914e+00, PNorm = 90.3917, GNorm = 301.0965, lr_0 = 7.3855e-04
Validation auc = 0.777465
Epoch 185
Validation auc = 0.775587
Epoch 186
Validation auc = 0.765258
Epoch 187
Validation auc = 0.754460
Epoch 188
Validation auc = 0.749061
Epoch 189
Loss = 1.2744e+00, PNorm = 90.7600, GNorm = 117.3615, lr_0 = 7.3224e-04
Validation auc = 0.739437
Epoch 190
Validation auc = 0.740610
Epoch 191
Validation auc = 0.744131
Epoch 192
Validation auc = 0.745775
Epoch 193
Validation auc = 0.749296
Epoch 194
Loss = 2.8545e+00, PNorm = 91.1538, GNorm = 283.7156, lr_0 = 7.2598e-04
Validation auc = 0.762911
Epoch 195
Validation auc = 0.771831
Epoch 196
Validation auc = 0.765023
Epoch 197
Validation auc = 0.744131
Epoch 198
Validation auc = 0.742254
Epoch 199
Loss = 2.9235e+00, PNorm = 91.5072, GNorm = 209.0575, lr_0 = 7.1977e-04
Validation auc = 0.752817
Epoch 200
Validation auc = 0.765258
Epoch 201
Validation auc = 0.759390
Epoch 202
Validation auc = 0.766901
Epoch 203
Validation auc = 0.770423
Epoch 204
Loss = 2.1208e+00, PNorm = 91.8952, GNorm = 282.7636, lr_0 = 7.1362e-04
Validation auc = 0.779577
Epoch 205
Validation auc = 0.781925
Epoch 206
Validation auc = 0.788028
Epoch 207
Validation auc = 0.786854
Epoch 208
Validation auc = 0.790141
Epoch 209
Loss = 2.0367e+00, PNorm = 92.2216, GNorm = 84.9593, lr_0 = 7.0752e-04
Validation auc = 0.780282
Epoch 210
Validation auc = 0.763615
Epoch 211
Validation auc = 0.760798
Epoch 212
Validation auc = 0.761737
Epoch 213
Validation auc = 0.776056
Epoch 214
Loss = 2.6354e+00, PNorm = 92.5181, GNorm = 171.0752, lr_0 = 7.0147e-04
Validation auc = 0.783099
Epoch 215
Validation auc = 0.782629
Epoch 216
Validation auc = 0.778404
Epoch 217
Validation auc = 0.758920
Epoch 218
Validation auc = 0.752582
Epoch 219
Loss = 1.5741e+00, PNorm = 92.8897, GNorm = 144.7433, lr_0 = 6.9548e-04
Validation auc = 0.752113
Epoch 220
Validation auc = 0.768545
Epoch 221
Validation auc = 0.762441
Epoch 222
Validation auc = 0.759155
Epoch 223
Validation auc = 0.763146
Epoch 224
Loss = 2.0410e+00, PNorm = 93.2505, GNorm = 190.2686, lr_0 = 6.8953e-04
Validation auc = 0.756808
Epoch 225
Validation auc = 0.757042
Epoch 226
Validation auc = 0.751174
Epoch 227
Validation auc = 0.749765
Epoch 228
Validation auc = 0.743192
Epoch 229
Loss = 2.1310e+00, PNorm = 93.5827, GNorm = 204.1606, lr_0 = 6.8364e-04
Validation auc = 0.738967
Epoch 230
Validation auc = 0.737559
Epoch 231
Validation auc = 0.740610
Epoch 232
Validation auc = 0.737324
Epoch 233
Validation auc = 0.737324
Epoch 234
Loss = 2.6351e+00, PNorm = 93.8499, GNorm = 168.6133, lr_0 = 6.7779e-04
Validation auc = 0.738498
Epoch 235
Validation auc = 0.739437
Epoch 236
Validation auc = 0.741080
Epoch 237
Validation auc = 0.742958
Epoch 238
Validation auc = 0.740141
Epoch 239
Loss = 2.2068e+00, PNorm = 94.1292, GNorm = 206.2297, lr_0 = 6.7200e-04
Validation auc = 0.730516
Epoch 240
Validation auc = 0.723474
Epoch 241
Validation auc = 0.718310
Epoch 242
Validation auc = 0.714319
Epoch 243
Validation auc = 0.726761
Epoch 244
Loss = 3.3219e+00, PNorm = 94.4586, GNorm = 157.0886, lr_0 = 6.6625e-04
Validation auc = 0.724648
Epoch 245
Validation auc = 0.728638
Epoch 246
Validation auc = 0.730516
Epoch 247
Validation auc = 0.735915
Epoch 248
Validation auc = 0.748357
Epoch 249
Loss = 1.6082e+00, PNorm = 94.8265, GNorm = 180.7270, lr_0 = 6.6056e-04
Validation auc = 0.756338
Epoch 250
Validation auc = 0.754930
Epoch 251
Validation auc = 0.747887
Epoch 252
Validation auc = 0.744601
Epoch 253
Validation auc = 0.732160
Epoch 254
Loss = 2.0290e+00, PNorm = 95.1869, GNorm = 244.6441, lr_0 = 6.5491e-04
Validation auc = 0.719718
Epoch 255
Validation auc = 0.715962
Epoch 256
Validation auc = 0.722066
Epoch 257
Validation auc = 0.724413
Epoch 258
Validation auc = 0.724648
Epoch 259
Loss = 5.9806e-01, PNorm = 95.4962, GNorm = 122.6883, lr_0 = 6.4931e-04
Validation auc = 0.730282
Epoch 260
Validation auc = 0.741080
Epoch 261
Validation auc = 0.749531
Epoch 262
Validation auc = 0.753521
Epoch 263
Validation auc = 0.763615
Epoch 264
Loss = 9.7261e-01, PNorm = 95.7920, GNorm = 76.8259, lr_0 = 6.4376e-04
Validation auc = 0.762676
Epoch 265
Validation auc = 0.760329
Epoch 266
Validation auc = 0.761268
Epoch 267
Validation auc = 0.759859
Epoch 268
Validation auc = 0.757746
Epoch 269
Loss = 1.6493e+00, PNorm = 96.0777, GNorm = 129.0162, lr_0 = 6.3826e-04
Validation auc = 0.757042
Epoch 270
Validation auc = 0.754695
Epoch 271
Validation auc = 0.747653
Epoch 272
Validation auc = 0.744601
Epoch 273
Validation auc = 0.743662
Epoch 274
Loss = 8.0202e-01, PNorm = 96.2823, GNorm = 81.5441, lr_0 = 6.3280e-04
Validation auc = 0.746009
Epoch 275
Validation auc = 0.746479
Epoch 276
Validation auc = 0.751643
Epoch 277
Validation auc = 0.751408
Epoch 278
Validation auc = 0.754930
Epoch 279
Loss = 1.6149e+00, PNorm = 96.4944, GNorm = 141.6787, lr_0 = 6.2739e-04
Validation auc = 0.757042
Epoch 280
Validation auc = 0.752113
Epoch 281
Validation auc = 0.743427
Epoch 282
Validation auc = 0.739906
Epoch 283
Validation auc = 0.748592
Epoch 284
Loss = 1.3580e+00, PNorm = 96.7686, GNorm = 150.1973, lr_0 = 6.2203e-04
Validation auc = 0.746244
Epoch 285
Validation auc = 0.739202
Epoch 286
Validation auc = 0.737324
Epoch 287
Validation auc = 0.743192
Epoch 288
Validation auc = 0.755634
Epoch 289
Loss = 6.7327e-01, PNorm = 97.0249, GNorm = 51.1246, lr_0 = 6.1671e-04
Validation auc = 0.756808
Epoch 290
Validation auc = 0.763615
Epoch 291
Validation auc = 0.776526
Epoch 292
Validation auc = 0.781221
Epoch 293
Validation auc = 0.789906
Epoch 294
Loss = 1.9053e+00, PNorm = 97.2242, GNorm = 126.3527, lr_0 = 6.1144e-04
Validation auc = 0.790610
Epoch 295
Validation auc = 0.791315
Epoch 296
Validation auc = 0.777934
Epoch 297
Validation auc = 0.774178
Epoch 298
Validation auc = 0.779812
Epoch 299
Loss = 1.2185e+00, PNorm = 97.4658, GNorm = 72.1986, lr_0 = 6.0621e-04
Validation auc = 0.784038
Model 0 best validation auc = 0.801174 on epoch 179
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.760084
Ensemble test auc = 0.760084
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.661737
Epoch 1
Validation auc = 0.712676
Epoch 2
Validation auc = 0.717840
Epoch 3
Validation auc = 0.716667
Epoch 4
Loss = 4.9900e+01, PNorm = 78.6964, GNorm = 969.6825, lr_0 = 6.5000e-04
Validation auc = 0.730516
Epoch 5
Validation auc = 0.712676
Epoch 6
Validation auc = 0.733803
Epoch 7
Validation auc = 0.713146
Epoch 8
Validation auc = 0.692488
Epoch 9
Loss = 3.8622e+01, PNorm = 79.0433, GNorm = 527.3305, lr_0 = 9.9743e-04
Validation auc = 0.677465
Epoch 10
Validation auc = 0.751408
Epoch 11
Validation auc = 0.654930
Epoch 12
Validation auc = 0.653521
Epoch 13
Validation auc = 0.727700
Epoch 14
Loss = 6.3268e+01, PNorm = 79.3988, GNorm = 1261.8366, lr_0 = 9.8890e-04
Validation auc = 0.706338
Epoch 15
Validation auc = 0.602817
Epoch 16
Validation auc = 0.555399
Epoch 17
Validation auc = 0.693897
Epoch 18
Validation auc = 0.635915
Epoch 19
Loss = 6.4142e+01, PNorm = 79.6875, GNorm = 670.4530, lr_0 = 9.8045e-04
Validation auc = 0.716197
Epoch 20
Validation auc = 0.621127
Epoch 21
Validation auc = 0.645070
Epoch 22
Validation auc = 0.718310
Epoch 23
Validation auc = 0.762441
Epoch 24
Loss = 3.4774e+01, PNorm = 79.8796, GNorm = 617.5802, lr_0 = 9.7207e-04
Validation auc = 0.725352
Epoch 25
Validation auc = 0.643192
Epoch 26
Validation auc = 0.608920
Epoch 27
Validation auc = 0.695540
Epoch 28
Validation auc = 0.744601
Epoch 29
Loss = 3.0644e+01, PNorm = 80.0188, GNorm = 601.9485, lr_0 = 9.6376e-04
Validation auc = 0.698592
Epoch 30
Validation auc = 0.574648
Epoch 31
Validation auc = 0.649061
Epoch 32
Validation auc = 0.716197
Epoch 33
Validation auc = 0.694366
Epoch 34
Loss = 2.4694e+01, PNorm = 80.1486, GNorm = 390.4154, lr_0 = 9.5552e-04
Validation auc = 0.597887
Epoch 35
Validation auc = 0.616901
Epoch 36
Validation auc = 0.694366
Epoch 37
Validation auc = 0.684272
Epoch 38
Validation auc = 0.647418
Epoch 39
Loss = 2.7164e+01, PNorm = 80.3584, GNorm = 1002.7186, lr_0 = 9.4735e-04
Validation auc = 0.592019
Epoch 40
Validation auc = 0.624883
Epoch 41
Validation auc = 0.736150
Epoch 42
Validation auc = 0.744131
Epoch 43
Validation auc = 0.706338
Epoch 44
Loss = 2.9101e+01, PNorm = 80.4640, GNorm = 708.3372, lr_0 = 9.3925e-04
Validation auc = 0.641315
Epoch 45
Validation auc = 0.678169
Epoch 46
Validation auc = 0.708685
Epoch 47
Validation auc = 0.707277
Epoch 48
Validation auc = 0.704695
Epoch 49
Loss = 1.5680e+01, PNorm = 80.6342, GNorm = 500.4720, lr_0 = 9.3122e-04
Validation auc = 0.722535
Epoch 50
Validation auc = 0.762207
Epoch 51
Validation auc = 0.750000
Epoch 52
Validation auc = 0.635211
Epoch 53
Validation auc = 0.672770
Epoch 54
Loss = 1.5721e+01, PNorm = 80.8014, GNorm = 353.2405, lr_0 = 9.2326e-04
Validation auc = 0.749765
Epoch 55
Validation auc = 0.735211
Epoch 56
Validation auc = 0.679343
Epoch 57
Validation auc = 0.659624
Epoch 58
Validation auc = 0.746948
Epoch 59
Loss = 2.4253e+01, PNorm = 81.0479, GNorm = 665.5268, lr_0 = 9.1537e-04
Validation auc = 0.743662
Epoch 60
Validation auc = 0.651408
Epoch 61
Validation auc = 0.616667
Epoch 62
Validation auc = 0.695775
Epoch 63
Validation auc = 0.753521
Epoch 64
Loss = 2.2211e+01, PNorm = 81.4007, GNorm = 696.7846, lr_0 = 9.0754e-04
Validation auc = 0.707981
Epoch 65
Validation auc = 0.646714
Epoch 66
Validation auc = 0.665023
Epoch 67
Validation auc = 0.692723
Epoch 68
Validation auc = 0.710329
Epoch 69
Loss = 3.1871e+01, PNorm = 81.8618, GNorm = 438.1842, lr_0 = 8.9978e-04
Validation auc = 0.693427
Epoch 70
Validation auc = 0.642488
Epoch 71
Validation auc = 0.650469
Epoch 72
Validation auc = 0.700235
Epoch 73
Validation auc = 0.709390
Epoch 74
Loss = 1.4393e+01, PNorm = 82.3698, GNorm = 855.8145, lr_0 = 8.9209e-04
Validation auc = 0.693662
Epoch 75
Validation auc = 0.688028
Epoch 76
Validation auc = 0.741549
Epoch 77
Validation auc = 0.762676
Epoch 78
Validation auc = 0.751878
Epoch 79
Loss = 1.1281e+01, PNorm = 82.7415, GNorm = 339.0408, lr_0 = 8.8447e-04
Validation auc = 0.666432
Epoch 80
Validation auc = 0.641080
Epoch 81
Validation auc = 0.715962
Epoch 82
Validation auc = 0.760798
Epoch 83
Validation auc = 0.762911
Epoch 84
Loss = 1.6464e+01, PNorm = 83.1255, GNorm = 653.6943, lr_0 = 8.7691e-04
Validation auc = 0.734977
Epoch 85
Validation auc = 0.671127
Epoch 86
Validation auc = 0.683099
Epoch 87
Validation auc = 0.729108
Epoch 88
Validation auc = 0.743192
Epoch 89
Loss = 1.4391e+01, PNorm = 83.6519, GNorm = 389.6971, lr_0 = 8.6941e-04
Validation auc = 0.750939
Epoch 90
Validation auc = 0.724883
Epoch 91
Validation auc = 0.659155
Epoch 92
Validation auc = 0.689202
Epoch 93
Validation auc = 0.729812
Epoch 94
Loss = 7.6764e+00, PNorm = 84.1371, GNorm = 294.7194, lr_0 = 8.6198e-04
Validation auc = 0.706338
Epoch 95
Validation auc = 0.659390
Epoch 96
Validation auc = 0.643662
Epoch 97
Validation auc = 0.669014
Epoch 98
Validation auc = 0.669718
Epoch 99
Loss = 8.0935e+00, PNorm = 84.4966, GNorm = 414.6125, lr_0 = 8.5461e-04
Validation auc = 0.690376
Epoch 100
Validation auc = 0.692254
Epoch 101
Validation auc = 0.679812
Epoch 102
Validation auc = 0.695540
Epoch 103
Validation auc = 0.716901
Epoch 104
Loss = 1.0548e+01, PNorm = 84.8684, GNorm = 480.7974, lr_0 = 8.4730e-04
Validation auc = 0.738732
Epoch 105
Validation auc = 0.743427
Epoch 106
Validation auc = 0.752347
Epoch 107
Validation auc = 0.769014
Epoch 108
Validation auc = 0.767840
Epoch 109
Loss = 8.7707e+00, PNorm = 85.2730, GNorm = 230.4737, lr_0 = 8.4006e-04
Validation auc = 0.765728
Epoch 110
Validation auc = 0.764554
Epoch 111
Validation auc = 0.760094
Epoch 112
Validation auc = 0.723474
Epoch 113
Validation auc = 0.716197
Epoch 114
Loss = 4.7525e+00, PNorm = 85.6340, GNorm = 217.4455, lr_0 = 8.3288e-04
Validation auc = 0.730047
Epoch 115
Validation auc = 0.743427
Epoch 116
Validation auc = 0.744366
Epoch 117
Validation auc = 0.729108
Epoch 118
Validation auc = 0.727465
Epoch 119
Loss = 5.6307e+00, PNorm = 86.0555, GNorm = 323.4030, lr_0 = 8.2576e-04
Validation auc = 0.751643
Epoch 120
Validation auc = 0.752113
Epoch 121
Validation auc = 0.770423
Epoch 122
Validation auc = 0.762911
Epoch 123
Validation auc = 0.748122
Epoch 124
Loss = 7.6575e+00, PNorm = 86.6643, GNorm = 695.3791, lr_0 = 8.1870e-04
Validation auc = 0.756338
Epoch 125
Validation auc = 0.759155
Epoch 126
Validation auc = 0.753521
Epoch 127
Validation auc = 0.750704
Epoch 128
Validation auc = 0.744601
Epoch 129
Loss = 7.6241e+00, PNorm = 87.0141, GNorm = 216.0523, lr_0 = 8.1170e-04
Validation auc = 0.734272
Epoch 130
Validation auc = 0.720657
Epoch 131
Validation auc = 0.715493
Epoch 132
Validation auc = 0.710563
Epoch 133
Validation auc = 0.696948
Epoch 134
Loss = 8.1833e+00, PNorm = 87.3464, GNorm = 349.2190, lr_0 = 8.0476e-04
Validation auc = 0.696948
Epoch 135
Validation auc = 0.697183
Epoch 136
Validation auc = 0.678169
Epoch 137
Validation auc = 0.680986
Epoch 138
Validation auc = 0.685915
Epoch 139
Loss = 3.9827e+00, PNorm = 87.8443, GNorm = 243.0568, lr_0 = 7.9788e-04
Validation auc = 0.692019
Epoch 140
Validation auc = 0.701174
Epoch 141
Validation auc = 0.699765
Epoch 142
Validation auc = 0.704695
Epoch 143
Validation auc = 0.722535
Epoch 144
Loss = 6.5270e+00, PNorm = 88.4850, GNorm = 261.3521, lr_0 = 7.9106e-04
Validation auc = 0.725587
Epoch 145
Validation auc = 0.726056
Epoch 146
Validation auc = 0.707981
Epoch 147
Validation auc = 0.710094
Epoch 148
Validation auc = 0.721596
Epoch 149
Loss = 7.3987e+00, PNorm = 88.9268, GNorm = 264.1017, lr_0 = 7.8430e-04
Validation auc = 0.723005
Epoch 150
Validation auc = 0.700235
Epoch 151
Validation auc = 0.690141
Epoch 152
Validation auc = 0.702113
Epoch 153
Validation auc = 0.711268
Epoch 154
Loss = 6.1881e+00, PNorm = 89.2553, GNorm = 267.3203, lr_0 = 7.7759e-04
Validation auc = 0.715962
Epoch 155
Validation auc = 0.721127
Epoch 156
Validation auc = 0.726761
Epoch 157
Validation auc = 0.742488
Epoch 158
Validation auc = 0.759390
Epoch 159
Loss = 4.5506e+00, PNorm = 89.7653, GNorm = 222.4000, lr_0 = 7.7095e-04
Validation auc = 0.761502
Epoch 160
Validation auc = 0.762207
Epoch 161
Validation auc = 0.751408
Epoch 162
Validation auc = 0.729812
Epoch 163
Validation auc = 0.723239
Epoch 164
Loss = 6.7666e+00, PNorm = 90.1993, GNorm = 233.9487, lr_0 = 7.6436e-04
Validation auc = 0.729108
Epoch 165
Validation auc = 0.729343
Epoch 166
Validation auc = 0.722300
Epoch 167
Validation auc = 0.710094
Epoch 168
Validation auc = 0.707512
Epoch 169
Loss = 7.9594e+00, PNorm = 90.5671, GNorm = 137.0509, lr_0 = 7.5782e-04
Validation auc = 0.730516
Epoch 170
Validation auc = 0.732394
Epoch 171
Validation auc = 0.738263
Epoch 172
Validation auc = 0.745775
Epoch 173
Validation auc = 0.728404
Epoch 174
Loss = 7.6934e+00, PNorm = 90.8960, GNorm = 331.3652, lr_0 = 7.5134e-04
Validation auc = 0.733803
Epoch 175
Validation auc = 0.746244
Epoch 176
Validation auc = 0.736385
Epoch 177
Validation auc = 0.717136
Epoch 178
Validation auc = 0.724648
Epoch 179
Loss = 1.7140e+00, PNorm = 91.1904, GNorm = 186.7137, lr_0 = 7.4492e-04
Validation auc = 0.732160
Epoch 180
Validation auc = 0.711033
Epoch 181
Validation auc = 0.728873
Epoch 182
Validation auc = 0.750235
Epoch 183
Validation auc = 0.752582
Epoch 184
Loss = 1.7925e+00, PNorm = 91.5690, GNorm = 114.1365, lr_0 = 7.3855e-04
Validation auc = 0.757042
Epoch 185
Validation auc = 0.755634
Epoch 186
Validation auc = 0.764319
Epoch 187
Validation auc = 0.773709
Epoch 188
Validation auc = 0.780282
Epoch 189
Loss = 4.7469e+00, PNorm = 91.8750, GNorm = 161.9434, lr_0 = 7.3224e-04
Validation auc = 0.783568
Epoch 190
Validation auc = 0.787324
Epoch 191
Validation auc = 0.787559
Epoch 192
Validation auc = 0.780282
Epoch 193
Validation auc = 0.768779
Epoch 194
Loss = 1.0567e+00, PNorm = 92.1886, GNorm = 84.4464, lr_0 = 7.2598e-04
Validation auc = 0.762441
Epoch 195
Validation auc = 0.759390
Epoch 196
Validation auc = 0.753991
Epoch 197
Validation auc = 0.747887
Epoch 198
Validation auc = 0.742488
Epoch 199
Loss = 4.7880e+00, PNorm = 92.5107, GNorm = 259.5184, lr_0 = 7.1977e-04
Validation auc = 0.746009
Epoch 200
Validation auc = 0.741315
Epoch 201
Validation auc = 0.735915
Epoch 202
Validation auc = 0.723474
Epoch 203
Validation auc = 0.716432
Epoch 204
Loss = 3.3348e+00, PNorm = 92.8167, GNorm = 221.6728, lr_0 = 7.1362e-04
Validation auc = 0.712676
Epoch 205
Validation auc = 0.709390
Epoch 206
Validation auc = 0.700939
Epoch 207
Validation auc = 0.693192
Epoch 208
Validation auc = 0.693897
Epoch 209
Loss = 2.3179e+00, PNorm = 93.1242, GNorm = 280.9472, lr_0 = 7.0752e-04
Validation auc = 0.689437
Epoch 210
Validation auc = 0.698826
Epoch 211
Validation auc = 0.694836
Epoch 212
Validation auc = 0.694131
Epoch 213
Validation auc = 0.703521
Epoch 214
Loss = 3.6084e+00, PNorm = 93.3629, GNorm = 223.8269, lr_0 = 7.0147e-04
Validation auc = 0.714085
Epoch 215
Validation auc = 0.716667
Epoch 216
Validation auc = 0.738732
Epoch 217
Validation auc = 0.733099
Epoch 218
Validation auc = 0.735681
Epoch 219
Loss = 9.5644e-01, PNorm = 93.6290, GNorm = 95.5903, lr_0 = 6.9548e-04
Validation auc = 0.739437
Epoch 220
Validation auc = 0.742723
Epoch 221
Validation auc = 0.746479
Epoch 222
Validation auc = 0.747418
Epoch 223
Validation auc = 0.743662
Epoch 224
Loss = 1.7011e+00, PNorm = 93.9247, GNorm = 214.4128, lr_0 = 6.8953e-04
Validation auc = 0.740610
Epoch 225
Validation auc = 0.740141
Epoch 226
Validation auc = 0.734038
Epoch 227
Validation auc = 0.734272
Epoch 228
Validation auc = 0.736150
Epoch 229
Loss = 1.6725e+00, PNorm = 94.2150, GNorm = 190.1290, lr_0 = 6.8364e-04
Validation auc = 0.735681
Epoch 230
Validation auc = 0.744131
Epoch 231
Validation auc = 0.751643
Epoch 232
Validation auc = 0.747418
Epoch 233
Validation auc = 0.749296
Epoch 234
Loss = 1.5988e+00, PNorm = 94.4271, GNorm = 183.9201, lr_0 = 6.7779e-04
Validation auc = 0.753521
Epoch 235
Validation auc = 0.755164
Epoch 236
Validation auc = 0.758920
Epoch 237
Validation auc = 0.759390
Epoch 238
Validation auc = 0.754225
Epoch 239
Loss = 2.8049e+00, PNorm = 94.6238, GNorm = 294.8777, lr_0 = 6.7200e-04
Validation auc = 0.746948
Epoch 240
Validation auc = 0.741549
Epoch 241
Validation auc = 0.740610
Epoch 242
Validation auc = 0.746479
Epoch 243
Validation auc = 0.748357
Epoch 244
Loss = 3.6799e+00, PNorm = 94.9052, GNorm = 449.9608, lr_0 = 6.6625e-04
Validation auc = 0.753521
Epoch 245
Validation auc = 0.752582
Epoch 246
Validation auc = 0.751643
Epoch 247
Validation auc = 0.749765
Epoch 248
Validation auc = 0.743427
Epoch 249
Loss = 1.8844e+00, PNorm = 95.1191, GNorm = 161.5968, lr_0 = 6.6056e-04
Validation auc = 0.722770
Epoch 250
Validation auc = 0.729812
Epoch 251
Validation auc = 0.756808
Epoch 252
Validation auc = 0.754695
Epoch 253
Validation auc = 0.747183
Epoch 254
Loss = 4.1293e+00, PNorm = 95.3310, GNorm = 155.5996, lr_0 = 6.5491e-04
Validation auc = 0.741080
Epoch 255
Validation auc = 0.731925
Epoch 256
Validation auc = 0.747887
Epoch 257
Validation auc = 0.768075
Epoch 258
Validation auc = 0.780516
Epoch 259
Loss = 2.7919e+00, PNorm = 95.6201, GNorm = 187.6118, lr_0 = 6.4931e-04
Validation auc = 0.780516
Epoch 260
Validation auc = 0.780751
Epoch 261
Validation auc = 0.766667
Epoch 262
Validation auc = 0.754930
Epoch 263
Validation auc = 0.744836
Epoch 264
Loss = 1.6727e+00, PNorm = 95.9612, GNorm = 281.7232, lr_0 = 6.4376e-04
Validation auc = 0.750704
Epoch 265
Validation auc = 0.769484
Epoch 266
Validation auc = 0.763850
Epoch 267
Validation auc = 0.749531
Epoch 268
Validation auc = 0.732394
Epoch 269
Loss = 1.0605e+00, PNorm = 96.1797, GNorm = 210.1468, lr_0 = 6.3826e-04
Validation auc = 0.732160
Epoch 270
Validation auc = 0.742723
Epoch 271
Validation auc = 0.735681
Epoch 272
Validation auc = 0.731925
Epoch 273
Validation auc = 0.725822
Epoch 274
Loss = 9.2520e-01, PNorm = 96.4101, GNorm = 90.4099, lr_0 = 6.3280e-04
Validation auc = 0.725587
Epoch 275
Validation auc = 0.727465
Epoch 276
Validation auc = 0.725352
Epoch 277
Validation auc = 0.730282
Epoch 278
Validation auc = 0.742723
Epoch 279
Loss = 1.2189e+00, PNorm = 96.5465, GNorm = 52.2598, lr_0 = 6.2739e-04
Validation auc = 0.751408
Epoch 280
Validation auc = 0.753286
Epoch 281
Validation auc = 0.733099
Epoch 282
Validation auc = 0.722535
Epoch 283
Validation auc = 0.737559
Epoch 284
Loss = 9.5200e-01, PNorm = 96.6844, GNorm = 97.6020, lr_0 = 6.2203e-04
Validation auc = 0.736385
Epoch 285
Validation auc = 0.730751
Epoch 286
Validation auc = 0.746009
Epoch 287
Validation auc = 0.752582
Epoch 288
Validation auc = 0.744131
Epoch 289
Loss = 1.6790e+00, PNorm = 96.8499, GNorm = 261.7581, lr_0 = 6.1671e-04
Validation auc = 0.750704
Epoch 290
Validation auc = 0.760094
Epoch 291
Validation auc = 0.760094
Epoch 292
Validation auc = 0.756573
Epoch 293
Validation auc = 0.748592
Epoch 294
Loss = 1.6235e+00, PNorm = 97.0757, GNorm = 71.6343, lr_0 = 6.1144e-04
Validation auc = 0.747653
Epoch 295
Validation auc = 0.752817
Epoch 296
Validation auc = 0.758920
Epoch 297
Validation auc = 0.766197
Epoch 298
Validation auc = 0.765023
Epoch 299
Loss = 7.7802e-01, PNorm = 97.1966, GNorm = 145.8290, lr_0 = 6.0621e-04
Validation auc = 0.767371
Model 0 best validation auc = 0.787559 on epoch 191
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.821493
Ensemble test auc = 0.821493
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.558216
Epoch 1
Validation auc = 0.653286
Epoch 2
Validation auc = 0.717840
Epoch 3
Validation auc = 0.703756
Epoch 4
Loss = 7.0150e+01, PNorm = 78.6994, GNorm = 1231.5674, lr_0 = 6.5000e-04
Validation auc = 0.688732
Epoch 5
Validation auc = 0.715023
Epoch 6
Validation auc = 0.714085
Epoch 7
Validation auc = 0.576995
Epoch 8
Validation auc = 0.730282
Epoch 9
Loss = 4.6842e+01, PNorm = 79.0090, GNorm = 1837.1336, lr_0 = 9.9743e-04
Validation auc = 0.709624
Epoch 10
Validation auc = 0.676056
Epoch 11
Validation auc = 0.654695
Epoch 12
Validation auc = 0.705399
Epoch 13
Validation auc = 0.736620
Epoch 14
Loss = 5.6569e+01, PNorm = 79.3492, GNorm = 1134.3087, lr_0 = 9.8890e-04
Validation auc = 0.692958
Epoch 15
Validation auc = 0.748826
Epoch 16
Validation auc = 0.715728
Epoch 17
Validation auc = 0.659390
Epoch 18
Validation auc = 0.709155
Epoch 19
Loss = 5.9651e+01, PNorm = 79.6050, GNorm = 1208.3800, lr_0 = 9.8045e-04
Validation auc = 0.726056
Epoch 20
Validation auc = 0.677465
Epoch 21
Validation auc = 0.684507
Epoch 22
Validation auc = 0.737559
Epoch 23
Validation auc = 0.738967
Epoch 24
Loss = 4.3781e+01, PNorm = 79.7873, GNorm = 1082.7551, lr_0 = 9.7207e-04
Validation auc = 0.627700
Epoch 25
Validation auc = 0.635446
Epoch 26
Validation auc = 0.693192
Epoch 27
Validation auc = 0.717840
Epoch 28
Validation auc = 0.727934
Epoch 29
Loss = 3.5038e+01, PNorm = 79.9638, GNorm = 526.1145, lr_0 = 9.6376e-04
Validation auc = 0.676761
Epoch 30
Validation auc = 0.692488
Epoch 31
Validation auc = 0.683099
Epoch 32
Validation auc = 0.675117
Epoch 33
Validation auc = 0.715023
Epoch 34
Loss = 3.8978e+01, PNorm = 80.1517, GNorm = 822.2265, lr_0 = 9.5552e-04
Validation auc = 0.684272
Epoch 35
Validation auc = 0.634507
Epoch 36
Validation auc = 0.693662
Epoch 37
Validation auc = 0.713146
Epoch 38
Validation auc = 0.734742
Epoch 39
Loss = 3.7598e+01, PNorm = 80.3460, GNorm = 1374.5530, lr_0 = 9.4735e-04
Validation auc = 0.685915
Epoch 40
Validation auc = 0.626526
Epoch 41
Validation auc = 0.719249
Epoch 42
Validation auc = 0.713380
Epoch 43
Validation auc = 0.638967
Epoch 44
Loss = 1.5134e+01, PNorm = 80.6087, GNorm = 932.0483, lr_0 = 9.3925e-04
Validation auc = 0.622066
Epoch 45
Validation auc = 0.731925
Epoch 46
Validation auc = 0.608216
Epoch 47
Validation auc = 0.746009
Epoch 48
Validation auc = 0.600939
Epoch 49
Loss = 6.6000e+01, PNorm = 80.9444, GNorm = 1648.4422, lr_0 = 9.3122e-04
Validation auc = 0.592958
Epoch 50
Validation auc = 0.751643
Epoch 51
Validation auc = 0.715493
Epoch 52
Validation auc = 0.752817
Epoch 53
Validation auc = 0.685681
Epoch 54
Loss = 3.0841e+01, PNorm = 81.3670, GNorm = 1478.3362, lr_0 = 9.2326e-04
Validation auc = 0.722535
Epoch 55
Validation auc = 0.735211
Epoch 56
Validation auc = 0.724883
Epoch 57
Validation auc = 0.727700
Epoch 58
Validation auc = 0.718310
Epoch 59
Loss = 2.5066e+01, PNorm = 81.7954, GNorm = 956.0689, lr_0 = 9.1537e-04
Validation auc = 0.723005
Epoch 60
Validation auc = 0.726056
Epoch 61
Validation auc = 0.724883
Epoch 62
Validation auc = 0.716432
Epoch 63
Validation auc = 0.692958
Epoch 64
Loss = 1.6881e+01, PNorm = 82.2850, GNorm = 592.0407, lr_0 = 9.0754e-04
Validation auc = 0.692488
Epoch 65
Validation auc = 0.699061
Epoch 66
Validation auc = 0.705634
Epoch 67
Validation auc = 0.702347
Epoch 68
Validation auc = 0.722770
Epoch 69
Loss = 2.0519e+01, PNorm = 82.6499, GNorm = 601.3826, lr_0 = 8.9978e-04
Validation auc = 0.753756
Epoch 70
Validation auc = 0.757746
Epoch 71
Validation auc = 0.725587
Epoch 72
Validation auc = 0.721596
Epoch 73
Validation auc = 0.739671
Epoch 74
Loss = 1.3884e+01, PNorm = 83.0393, GNorm = 569.3688, lr_0 = 8.9209e-04
Validation auc = 0.739202
Epoch 75
Validation auc = 0.710329
Epoch 76
Validation auc = 0.689671
Epoch 77
Validation auc = 0.720657
Epoch 78
Validation auc = 0.709390
Epoch 79
Loss = 1.8698e+01, PNorm = 83.4514, GNorm = 323.1243, lr_0 = 8.8447e-04
Validation auc = 0.747183
Epoch 80
Validation auc = 0.761737
Epoch 81
Validation auc = 0.737089
Epoch 82
Validation auc = 0.689202
Epoch 83
Validation auc = 0.671127
Epoch 84
Loss = 1.7675e+01, PNorm = 83.9922, GNorm = 592.6336, lr_0 = 8.7691e-04
Validation auc = 0.717136
Epoch 85
Validation auc = 0.737793
Epoch 86
Validation auc = 0.711033
Epoch 87
Validation auc = 0.671596
Epoch 88
Validation auc = 0.662207
Epoch 89
Loss = 1.5925e+01, PNorm = 84.4302, GNorm = 462.9757, lr_0 = 8.6941e-04
Validation auc = 0.733803
Epoch 90
Validation auc = 0.758451
Epoch 91
Validation auc = 0.746244
Epoch 92
Validation auc = 0.694131
Epoch 93
Validation auc = 0.649531
Epoch 94
Loss = 1.3650e+01, PNorm = 84.8136, GNorm = 1096.4494, lr_0 = 8.6198e-04
Validation auc = 0.740141
Epoch 95
Validation auc = 0.750704
Epoch 96
Validation auc = 0.750704
Epoch 97
Validation auc = 0.751643
Epoch 98
Validation auc = 0.725822
Epoch 99
Loss = 1.0321e+01, PNorm = 85.1864, GNorm = 275.3043, lr_0 = 8.5461e-04
Validation auc = 0.696714
Epoch 100
Validation auc = 0.703756
Epoch 101
Validation auc = 0.763615
Epoch 102
Validation auc = 0.778638
Epoch 103
Validation auc = 0.776291
Epoch 104
Loss = 6.9070e+00, PNorm = 85.8087, GNorm = 410.7169, lr_0 = 8.4730e-04
Validation auc = 0.745070
Epoch 105
Validation auc = 0.697418
Epoch 106
Validation auc = 0.736385
Epoch 107
Validation auc = 0.776291
Epoch 108
Validation auc = 0.762911
Epoch 109
Loss = 6.5732e+00, PNorm = 86.2679, GNorm = 272.8240, lr_0 = 8.4006e-04
Validation auc = 0.775117
Epoch 110
Validation auc = 0.759390
Epoch 111
Validation auc = 0.746009
Epoch 112
Validation auc = 0.762676
Epoch 113
Validation auc = 0.765023
Epoch 114
Loss = 4.4259e+00, PNorm = 86.8035, GNorm = 624.1233, lr_0 = 8.3288e-04
Validation auc = 0.761502
Epoch 115
Validation auc = 0.772770
Epoch 116
Validation auc = 0.784507
Epoch 117
Validation auc = 0.788732
Epoch 118
Validation auc = 0.782394
Epoch 119
Loss = 1.4996e+01, PNorm = 87.4410, GNorm = 514.2320, lr_0 = 8.2576e-04
Validation auc = 0.776995
Epoch 120
Validation auc = 0.774883
Epoch 121
Validation auc = 0.760798
Epoch 122
Validation auc = 0.765493
Epoch 123
Validation auc = 0.773005
Epoch 124
Loss = 4.5997e+00, PNorm = 87.9333, GNorm = 227.2685, lr_0 = 8.1870e-04
Validation auc = 0.773005
Epoch 125
Validation auc = 0.759624
Epoch 126
Validation auc = 0.730751
Epoch 127
Validation auc = 0.720657
Epoch 128
Validation auc = 0.721127
Epoch 129
Loss = 3.3091e+00, PNorm = 88.3274, GNorm = 241.3632, lr_0 = 8.1170e-04
Validation auc = 0.723709
Epoch 130
Validation auc = 0.719484
Epoch 131
Validation auc = 0.705399
Epoch 132
Validation auc = 0.680516
Epoch 133
Validation auc = 0.667606
Epoch 134
Loss = 1.0086e+01, PNorm = 88.7017, GNorm = 483.2506, lr_0 = 8.0476e-04
Validation auc = 0.685915
Epoch 135
Validation auc = 0.732629
Epoch 136
Validation auc = 0.733099
Epoch 137
Validation auc = 0.702817
Epoch 138
Validation auc = 0.695305
Epoch 139
Loss = 4.4733e+00, PNorm = 89.2274, GNorm = 262.8855, lr_0 = 7.9788e-04
Validation auc = 0.706103
Epoch 140
Validation auc = 0.715728
Epoch 141
Validation auc = 0.758451
Epoch 142
Validation auc = 0.763146
Epoch 143
Validation auc = 0.753756
Epoch 144
Loss = 5.9380e+00, PNorm = 89.7809, GNorm = 369.1171, lr_0 = 7.9106e-04
Validation auc = 0.711268
Epoch 145
Validation auc = 0.729343
Epoch 146
Validation auc = 0.748122
Epoch 147
Validation auc = 0.755869
Epoch 148
Validation auc = 0.759390
Epoch 149
Loss = 1.1238e+01, PNorm = 90.1559, GNorm = 342.2817, lr_0 = 7.8430e-04
Validation auc = 0.759624
Epoch 150
Validation auc = 0.747418
Epoch 151
Validation auc = 0.733333
Epoch 152
Validation auc = 0.715023
Epoch 153
Validation auc = 0.712207
Epoch 154
Loss = 6.1871e+00, PNorm = 90.6040, GNorm = 238.3828, lr_0 = 7.7759e-04
Validation auc = 0.724648
Epoch 155
Validation auc = 0.735446
Epoch 156
Validation auc = 0.736150
Epoch 157
Validation auc = 0.729108
Epoch 158
Validation auc = 0.756573
Epoch 159
Loss = 8.5698e+00, PNorm = 90.9477, GNorm = 621.1407, lr_0 = 7.7095e-04
Validation auc = 0.786620
Epoch 160
Validation auc = 0.787559
Epoch 161
Validation auc = 0.799531
Epoch 162
Validation auc = 0.790845
Epoch 163
Validation auc = 0.757042
Epoch 164
Loss = 4.5155e+00, PNorm = 91.3226, GNorm = 273.7544, lr_0 = 7.6436e-04
Validation auc = 0.740141
Epoch 165
Validation auc = 0.749061
Epoch 166
Validation auc = 0.771127
Epoch 167
Validation auc = 0.768779
Epoch 168
Validation auc = 0.759155
Epoch 169
Loss = 3.0917e+00, PNorm = 91.6136, GNorm = 512.9666, lr_0 = 7.5782e-04
Validation auc = 0.759624
Epoch 170
Validation auc = 0.751174
Epoch 171
Validation auc = 0.746009
Epoch 172
Validation auc = 0.743192
Epoch 173
Validation auc = 0.729577
Epoch 174
Loss = 4.3240e+00, PNorm = 91.8604, GNorm = 356.4250, lr_0 = 7.5134e-04
Validation auc = 0.733803
Epoch 175
Validation auc = 0.731455
Epoch 176
Validation auc = 0.734507
Epoch 177
Validation auc = 0.746244
Epoch 178
Validation auc = 0.757042
Epoch 179
Loss = 3.2355e+00, PNorm = 92.0949, GNorm = 186.6529, lr_0 = 7.4492e-04
Validation auc = 0.755164
Epoch 180
Validation auc = 0.763146
Epoch 181
Validation auc = 0.765728
Epoch 182
Validation auc = 0.762441
Epoch 183
Validation auc = 0.759155
Epoch 184
Loss = 3.8258e+00, PNorm = 92.3535, GNorm = 162.6064, lr_0 = 7.3855e-04
Validation auc = 0.736150
Epoch 185
Validation auc = 0.730986
Epoch 186
Validation auc = 0.725822
Epoch 187
Validation auc = 0.714554
Epoch 188
Validation auc = 0.726526
Epoch 189
Loss = 2.7522e+00, PNorm = 92.6970, GNorm = 46.7358, lr_0 = 7.3224e-04
Validation auc = 0.733099
Epoch 190
Validation auc = 0.742958
Epoch 191
Validation auc = 0.744366
Epoch 192
Validation auc = 0.744366
Epoch 193
Validation auc = 0.746244
Epoch 194
Loss = 6.3008e+00, PNorm = 92.9921, GNorm = 239.5995, lr_0 = 7.2598e-04
Validation auc = 0.758451
Epoch 195
Validation auc = 0.757746
Epoch 196
Validation auc = 0.757277
Epoch 197
Validation auc = 0.757512
Epoch 198
Validation auc = 0.746244
Epoch 199
Loss = 1.0166e+00, PNorm = 93.3022, GNorm = 166.5249, lr_0 = 7.1977e-04
Validation auc = 0.742254
Epoch 200
Validation auc = 0.752817
Epoch 201
Validation auc = 0.742019
Epoch 202
Validation auc = 0.734038
Epoch 203
Validation auc = 0.733803
Epoch 204
Loss = 1.4519e+00, PNorm = 93.5512, GNorm = 182.4309, lr_0 = 7.1362e-04
Validation auc = 0.752582
Epoch 205
Validation auc = 0.770188
Epoch 206
Validation auc = 0.778404
Epoch 207
Validation auc = 0.775822
Epoch 208
Validation auc = 0.778638
Epoch 209
Loss = 2.4677e+00, PNorm = 93.8217, GNorm = 152.1123, lr_0 = 7.0752e-04
Validation auc = 0.773005
Epoch 210
Validation auc = 0.757277
Epoch 211
Validation auc = 0.755164
Epoch 212
Validation auc = 0.757512
Epoch 213
Validation auc = 0.757746
Epoch 214
Loss = 2.1617e+00, PNorm = 94.0792, GNorm = 95.4994, lr_0 = 7.0147e-04
Validation auc = 0.753052
Epoch 215
Validation auc = 0.756808
Epoch 216
Validation auc = 0.763380
Epoch 217
Validation auc = 0.777465
Epoch 218
Validation auc = 0.779577
Epoch 219
Loss = 2.0433e+00, PNorm = 94.3150, GNorm = 131.6465, lr_0 = 6.9548e-04
Validation auc = 0.775587
Epoch 220
Validation auc = 0.777230
Epoch 221
Validation auc = 0.774413
Epoch 222
Validation auc = 0.776056
Epoch 223
Validation auc = 0.772300
Epoch 224
Loss = 3.0340e+00, PNorm = 94.6851, GNorm = 156.4080, lr_0 = 6.8953e-04
Validation auc = 0.766901
Epoch 225
Validation auc = 0.750704
Epoch 226
Validation auc = 0.736620
Epoch 227
Validation auc = 0.720423
Epoch 228
Validation auc = 0.691784
Epoch 229
Loss = 1.1868e+00, PNorm = 95.1289, GNorm = 99.0182, lr_0 = 6.8364e-04
Validation auc = 0.676056
Epoch 230
Validation auc = 0.684977
Epoch 231
Validation auc = 0.694836
Epoch 232
Validation auc = 0.710563
Epoch 233
Validation auc = 0.710563
Epoch 234
Loss = 2.0551e+00, PNorm = 95.5497, GNorm = 65.9067, lr_0 = 6.7779e-04
Validation auc = 0.708216
Epoch 235
Validation auc = 0.700939
Epoch 236
Validation auc = 0.700000
Epoch 237
Validation auc = 0.696714
Epoch 238
Validation auc = 0.696714
Epoch 239
Loss = 1.5195e+00, PNorm = 95.9174, GNorm = 260.8156, lr_0 = 6.7200e-04
Validation auc = 0.698826
Epoch 240
Validation auc = 0.702582
Epoch 241
Validation auc = 0.712441
Epoch 242
Validation auc = 0.718545
Epoch 243
Validation auc = 0.727230
Epoch 244
Loss = 2.1627e+00, PNorm = 96.2427, GNorm = 251.0288, lr_0 = 6.6625e-04
Validation auc = 0.738028
Epoch 245
Validation auc = 0.745540
Epoch 246
Validation auc = 0.747418
Epoch 247
Validation auc = 0.752582
Epoch 248
Validation auc = 0.761268
Epoch 249
Loss = 1.6189e+00, PNorm = 96.5365, GNorm = 115.2894, lr_0 = 6.6056e-04
Validation auc = 0.758216
Epoch 250
Validation auc = 0.762207
Epoch 251
Validation auc = 0.755869
Epoch 252
Validation auc = 0.756338
Epoch 253
Validation auc = 0.754460
Epoch 254
Loss = 1.4311e+00, PNorm = 96.8011, GNorm = 219.5703, lr_0 = 6.5491e-04
Validation auc = 0.731690
Epoch 255
Validation auc = 0.721831
Epoch 256
Validation auc = 0.720892
Epoch 257
Validation auc = 0.737324
Epoch 258
Validation auc = 0.763380
Epoch 259
Loss = 2.6747e+00, PNorm = 97.0166, GNorm = 165.7966, lr_0 = 6.4931e-04
Validation auc = 0.776526
Epoch 260
Validation auc = 0.784038
Epoch 261
Validation auc = 0.787324
Epoch 262
Validation auc = 0.779812
Epoch 263
Validation auc = 0.778169
Epoch 264
Loss = 4.7283e-01, PNorm = 97.3063, GNorm = 46.5081, lr_0 = 6.4376e-04
Validation auc = 0.774413
Epoch 265
Validation auc = 0.769249
Epoch 266
Validation auc = 0.749531
Epoch 267
Validation auc = 0.737324
Epoch 268
Validation auc = 0.741549
Epoch 269
Loss = 1.4192e+00, PNorm = 97.6300, GNorm = 108.9683, lr_0 = 6.3826e-04
Validation auc = 0.744131
Epoch 270
Validation auc = 0.750000
Epoch 271
Validation auc = 0.735915
Epoch 272
Validation auc = 0.719953
Epoch 273
Validation auc = 0.706338
Epoch 274
Loss = 9.1908e-01, PNorm = 97.9548, GNorm = 81.2718, lr_0 = 6.3280e-04
Validation auc = 0.698826
Epoch 275
Validation auc = 0.696479
Epoch 276
Validation auc = 0.698357
Epoch 277
Validation auc = 0.705399
Epoch 278
Validation auc = 0.698122
Epoch 279
Loss = 8.4772e-01, PNorm = 98.2483, GNorm = 57.6644, lr_0 = 6.2739e-04
Validation auc = 0.706573
Epoch 280
Validation auc = 0.710798
Epoch 281
Validation auc = 0.695540
Epoch 282
Validation auc = 0.695070
Epoch 283
Validation auc = 0.700469
Epoch 284
Loss = 1.6875e+00, PNorm = 98.4667, GNorm = 62.7192, lr_0 = 6.2203e-04
Validation auc = 0.705869
Epoch 285
Validation auc = 0.724883
Epoch 286
Validation auc = 0.724883
Epoch 287
Validation auc = 0.741080
Epoch 288
Validation auc = 0.744366
Epoch 289
Loss = 5.4483e-01, PNorm = 98.6283, GNorm = 89.9772, lr_0 = 6.1671e-04
Validation auc = 0.752113
Epoch 290
Validation auc = 0.753286
Epoch 291
Validation auc = 0.753756
Epoch 292
Validation auc = 0.753521
Epoch 293
Validation auc = 0.751878
Epoch 294
Loss = 7.3937e-01, PNorm = 98.8410, GNorm = 252.4158, lr_0 = 6.1144e-04
Validation auc = 0.750000
Epoch 295
Validation auc = 0.745070
Epoch 296
Validation auc = 0.740141
Epoch 297
Validation auc = 0.734038
Epoch 298
Validation auc = 0.733099
Epoch 299
Loss = 9.3824e-01, PNorm = 99.1080, GNorm = 110.6230, lr_0 = 6.0621e-04
Validation auc = 0.723239
Model 0 best validation auc = 0.799531 on epoch 161
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.756773
Ensemble test auc = 0.756773
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.623005
Epoch 1
Validation auc = 0.593897
Epoch 2
Validation auc = 0.704695
Epoch 3
Validation auc = 0.725822
Epoch 4
Loss = 4.4357e+01, PNorm = 78.6836, GNorm = 603.1781, lr_0 = 6.5000e-04
Validation auc = 0.720423
Epoch 5
Validation auc = 0.712207
Epoch 6
Validation auc = 0.686620
Epoch 7
Validation auc = 0.717606
Epoch 8
Validation auc = 0.626761
Epoch 9
Loss = 3.8704e+01, PNorm = 78.9716, GNorm = 679.1711, lr_0 = 9.9743e-04
Validation auc = 0.692019
Epoch 10
Validation auc = 0.650939
Epoch 11
Validation auc = 0.665258
Epoch 12
Validation auc = 0.620188
Epoch 13
Validation auc = 0.613615
Epoch 14
Loss = 5.0196e+01, PNorm = 79.2566, GNorm = 1375.8277, lr_0 = 9.8890e-04
Validation auc = 0.665728
Epoch 15
Validation auc = 0.554460
Epoch 16
Validation auc = 0.698122
Epoch 17
Validation auc = 0.694366
Epoch 18
Validation auc = 0.678404
Epoch 19
Loss = 4.6995e+01, PNorm = 79.4612, GNorm = 1397.4770, lr_0 = 9.8045e-04
Validation auc = 0.656338
Epoch 20
Validation auc = 0.664319
Epoch 21
Validation auc = 0.638967
Epoch 22
Validation auc = 0.730282
Epoch 23
Validation auc = 0.732160
Epoch 24
Loss = 4.9040e+01, PNorm = 79.6098, GNorm = 1135.7561, lr_0 = 9.7207e-04
Validation auc = 0.643192
Epoch 25
Validation auc = 0.629577
Epoch 26
Validation auc = 0.728404
Epoch 27
Validation auc = 0.710798
Epoch 28
Validation auc = 0.673239
Epoch 29
Loss = 3.0574e+01, PNorm = 79.7270, GNorm = 983.3814, lr_0 = 9.6376e-04
Validation auc = 0.665728
Epoch 30
Validation auc = 0.700469
Epoch 31
Validation auc = 0.742254
Epoch 32
Validation auc = 0.687793
Epoch 33
Validation auc = 0.680986
Epoch 34
Loss = 1.2164e+01, PNorm = 79.8490, GNorm = 607.9022, lr_0 = 9.5552e-04
Validation auc = 0.721127
Epoch 35
Validation auc = 0.716432
Epoch 36
Validation auc = 0.707512
Epoch 37
Validation auc = 0.711737
Epoch 38
Validation auc = 0.733099
Epoch 39
Loss = 3.4834e+01, PNorm = 79.9831, GNorm = 1061.8998, lr_0 = 9.4735e-04
Validation auc = 0.681455
Epoch 40
Validation auc = 0.625117
Epoch 41
Validation auc = 0.715023
Epoch 42
Validation auc = 0.724413
Epoch 43
Validation auc = 0.698122
Epoch 44
Loss = 4.0392e+01, PNorm = 80.1494, GNorm = 468.5277, lr_0 = 9.3925e-04
Validation auc = 0.695305
Epoch 45
Validation auc = 0.706338
Epoch 46
Validation auc = 0.704695
Epoch 47
Validation auc = 0.705869
Epoch 48
Validation auc = 0.692723
Epoch 49
Loss = 1.7215e+01, PNorm = 80.3478, GNorm = 430.5713, lr_0 = 9.3122e-04
Validation auc = 0.708685
Epoch 50
Validation auc = 0.707512
Epoch 51
Validation auc = 0.709859
Epoch 52
Validation auc = 0.700469
Epoch 53
Validation auc = 0.656338
Epoch 54
Loss = 1.9446e+01, PNorm = 80.5564, GNorm = 687.9145, lr_0 = 9.2326e-04
Validation auc = 0.719249
Epoch 55
Validation auc = 0.729108
Epoch 56
Validation auc = 0.680516
Epoch 57
Validation auc = 0.657981
Epoch 58
Validation auc = 0.689671
Epoch 59
Loss = 1.4108e+01, PNorm = 80.8077, GNorm = 265.5969, lr_0 = 9.1537e-04
Validation auc = 0.734742
Epoch 60
Validation auc = 0.768075
Epoch 61
Validation auc = 0.720892
Epoch 62
Validation auc = 0.681455
Epoch 63
Validation auc = 0.685211
Epoch 64
Loss = 3.5031e+01, PNorm = 81.1072, GNorm = 363.7206, lr_0 = 9.0754e-04
Validation auc = 0.731690
Epoch 65
Validation auc = 0.749296
Epoch 66
Validation auc = 0.740376
Epoch 67
Validation auc = 0.690376
Epoch 68
Validation auc = 0.684038
Epoch 69
Loss = 1.6178e+01, PNorm = 81.4954, GNorm = 597.9670, lr_0 = 8.9978e-04
Validation auc = 0.751174
Epoch 70
Validation auc = 0.733803
Epoch 71
Validation auc = 0.739906
Epoch 72
Validation auc = 0.734272
Epoch 73
Validation auc = 0.733333
Epoch 74
Loss = 1.7154e+01, PNorm = 82.0358, GNorm = 398.5587, lr_0 = 8.9209e-04
Validation auc = 0.712911
Epoch 75
Validation auc = 0.693427
Epoch 76
Validation auc = 0.715023
Epoch 77
Validation auc = 0.716901
Epoch 78
Validation auc = 0.718779
Epoch 79
Loss = 1.0310e+01, PNorm = 82.5273, GNorm = 265.3844, lr_0 = 8.8447e-04
Validation auc = 0.703756
Epoch 80
Validation auc = 0.680751
Epoch 81
Validation auc = 0.683333
Epoch 82
Validation auc = 0.725352
Epoch 83
Validation auc = 0.754460
Epoch 84
Loss = 1.5061e+01, PNorm = 83.0112, GNorm = 415.3190, lr_0 = 8.7691e-04
Validation auc = 0.753756
Epoch 85
Validation auc = 0.725822
Epoch 86
Validation auc = 0.705634
Epoch 87
Validation auc = 0.719014
Epoch 88
Validation auc = 0.757746
Epoch 89
Loss = 1.2933e+01, PNorm = 83.5251, GNorm = 572.7084, lr_0 = 8.6941e-04
Validation auc = 0.765258
Epoch 90
Validation auc = 0.747887
Epoch 91
Validation auc = 0.716432
Epoch 92
Validation auc = 0.698357
Epoch 93
Validation auc = 0.708451
Epoch 94
Loss = 1.5864e+01, PNorm = 83.9854, GNorm = 478.8795, lr_0 = 8.6198e-04
Validation auc = 0.726995
Epoch 95
Validation auc = 0.728873
Epoch 96
Validation auc = 0.713850
Epoch 97
Validation auc = 0.711972
Epoch 98
Validation auc = 0.689906
Epoch 99
Loss = 1.0234e+01, PNorm = 84.6109, GNorm = 312.5032, lr_0 = 8.5461e-04
Validation auc = 0.666197
Epoch 100
Validation auc = 0.703991
Epoch 101
Validation auc = 0.738732
Epoch 102
Validation auc = 0.734038
Epoch 103
Validation auc = 0.684742
Epoch 104
Loss = 8.9354e+00, PNorm = 85.2058, GNorm = 436.8983, lr_0 = 8.4730e-04
Validation auc = 0.671831
Epoch 105
Validation auc = 0.695540
Epoch 106
Validation auc = 0.748592
Epoch 107
Validation auc = 0.760563
Epoch 108
Validation auc = 0.747183
Epoch 109
Loss = 5.0284e+00, PNorm = 86.0167, GNorm = 221.7831, lr_0 = 8.4006e-04
Validation auc = 0.704695
Epoch 110
Validation auc = 0.758216
Epoch 111
Validation auc = 0.768545
Epoch 112
Validation auc = 0.775352
Epoch 113
Validation auc = 0.707042
Epoch 114
Loss = 4.9168e+00, PNorm = 86.5682, GNorm = 358.3475, lr_0 = 8.3288e-04
Validation auc = 0.681690
Epoch 115
Validation auc = 0.720892
Epoch 116
Validation auc = 0.732629
Epoch 117
Validation auc = 0.737324
Epoch 118
Validation auc = 0.740845
Epoch 119
Loss = 5.4940e+00, PNorm = 87.2107, GNorm = 420.3253, lr_0 = 8.2576e-04
Validation auc = 0.741549
Epoch 120
Validation auc = 0.714554
Epoch 121
Validation auc = 0.699061
Epoch 122
Validation auc = 0.699765
Epoch 123
Validation auc = 0.697418
Epoch 124
Loss = 2.8925e+00, PNorm = 87.7367, GNorm = 238.1425, lr_0 = 8.1870e-04
Validation auc = 0.691784
Epoch 125
Validation auc = 0.699296
Epoch 126
Validation auc = 0.713146
Epoch 127
Validation auc = 0.739437
Epoch 128
Validation auc = 0.730986
Epoch 129
Loss = 8.6627e+00, PNorm = 88.3141, GNorm = 342.5109, lr_0 = 8.1170e-04
Validation auc = 0.731455
Epoch 130
Validation auc = 0.755634
Epoch 131
Validation auc = 0.769484
Epoch 132
Validation auc = 0.768075
Epoch 133
Validation auc = 0.757512
Epoch 134
Loss = 5.6243e+00, PNorm = 88.8642, GNorm = 279.3522, lr_0 = 8.0476e-04
Validation auc = 0.713850
Epoch 135
Validation auc = 0.714319
Epoch 136
Validation auc = 0.751174
Epoch 137
Validation auc = 0.765258
Epoch 138
Validation auc = 0.747887
Epoch 139
Loss = 4.9113e+00, PNorm = 89.3152, GNorm = 158.6953, lr_0 = 7.9788e-04
Validation auc = 0.729108
Epoch 140
Validation auc = 0.724413
Epoch 141
Validation auc = 0.709859
Epoch 142
Validation auc = 0.710798
Epoch 143
Validation auc = 0.725822
Epoch 144
Loss = 7.7632e+00, PNorm = 89.8147, GNorm = 407.8018, lr_0 = 7.9106e-04
Validation auc = 0.750704
Epoch 145
Validation auc = 0.768310
Epoch 146
Validation auc = 0.750704
Epoch 147
Validation auc = 0.697183
Epoch 148
Validation auc = 0.695775
Epoch 149
Loss = 8.6087e+00, PNorm = 90.3933, GNorm = 325.6410, lr_0 = 7.8430e-04
Validation auc = 0.744366
Epoch 150
Validation auc = 0.753991
Epoch 151
Validation auc = 0.746714
Epoch 152
Validation auc = 0.718075
Epoch 153
Validation auc = 0.703521
Epoch 154
Loss = 3.3665e+00, PNorm = 90.8460, GNorm = 129.5773, lr_0 = 7.7759e-04
Validation auc = 0.698592
Epoch 155
Validation auc = 0.706338
Epoch 156
Validation auc = 0.722300
Epoch 157
Validation auc = 0.726526
Epoch 158
Validation auc = 0.720188
Epoch 159
Loss = 1.7238e+00, PNorm = 91.2440, GNorm = 304.2396, lr_0 = 7.7095e-04
Validation auc = 0.686385
Epoch 160
Validation auc = 0.662441
Epoch 161
Validation auc = 0.672535
Epoch 162
Validation auc = 0.699765
Epoch 163
Validation auc = 0.726291
Epoch 164
Loss = 5.3828e+00, PNorm = 91.6730, GNorm = 282.0232, lr_0 = 7.6436e-04
Validation auc = 0.729108
Epoch 165
Validation auc = 0.711972
Epoch 166
Validation auc = 0.708216
Epoch 167
Validation auc = 0.710329
Epoch 168
Validation auc = 0.702347
Epoch 169
Loss = 7.5173e-01, PNorm = 92.0926, GNorm = 41.8895, lr_0 = 7.5782e-04
Validation auc = 0.696479
Epoch 170
Validation auc = 0.681925
Epoch 171
Validation auc = 0.686385
Epoch 172
Validation auc = 0.719718
Epoch 173
Validation auc = 0.749061
Epoch 174
Loss = 3.9897e+00, PNorm = 92.4638, GNorm = 320.8694, lr_0 = 7.5134e-04
Validation auc = 0.751878
Epoch 175
Validation auc = 0.768545
Epoch 176
Validation auc = 0.760329
Epoch 177
Validation auc = 0.718075
Epoch 178
Validation auc = 0.711268
Epoch 179
Loss = 3.6578e+00, PNorm = 92.8602, GNorm = 81.8358, lr_0 = 7.4492e-04
Validation auc = 0.749765
Epoch 180
Validation auc = 0.770657
Epoch 181
Validation auc = 0.768310
Epoch 182
Validation auc = 0.753521
Epoch 183
Validation auc = 0.752347
Epoch 184
Loss = 3.4749e+00, PNorm = 93.2360, GNorm = 119.8474, lr_0 = 7.3855e-04
Validation auc = 0.736150
Epoch 185
Validation auc = 0.728169
Epoch 186
Validation auc = 0.712441
Epoch 187
Validation auc = 0.707277
Epoch 188
Validation auc = 0.703052
Epoch 189
Loss = 4.4865e+00, PNorm = 93.5601, GNorm = 175.4696, lr_0 = 7.3224e-04
Validation auc = 0.665728
Epoch 190
Validation auc = 0.670188
Epoch 191
Validation auc = 0.681925
Epoch 192
Validation auc = 0.726995
Epoch 193
Validation auc = 0.761033
Epoch 194
Loss = 4.6862e+00, PNorm = 93.7816, GNorm = 380.5779, lr_0 = 7.2598e-04
Validation auc = 0.751408
Epoch 195
Validation auc = 0.759624
Epoch 196
Validation auc = 0.758920
Epoch 197
Validation auc = 0.755399
Epoch 198
Validation auc = 0.766197
Epoch 199
Loss = 3.5574e+00, PNorm = 94.0904, GNorm = 262.6447, lr_0 = 7.1977e-04
Validation auc = 0.773944
Epoch 200
Validation auc = 0.769249
Epoch 201
Validation auc = 0.768779
Epoch 202
Validation auc = 0.765258
Epoch 203
Validation auc = 0.767606
Epoch 204
Loss = 3.1142e+00, PNorm = 94.4076, GNorm = 163.4307, lr_0 = 7.1362e-04
Validation auc = 0.772066
Epoch 205
Validation auc = 0.779812
Epoch 206
Validation auc = 0.768075
Epoch 207
Validation auc = 0.763615
Epoch 208
Validation auc = 0.739671
Epoch 209
Loss = 2.9367e+00, PNorm = 94.6269, GNorm = 151.7318, lr_0 = 7.0752e-04
Validation auc = 0.742723
Epoch 210
Validation auc = 0.744131
Epoch 211
Validation auc = 0.750469
Epoch 212
Validation auc = 0.792019
Epoch 213
Validation auc = 0.758685
Epoch 214
Loss = 2.4934e+00, PNorm = 94.9061, GNorm = 102.7062, lr_0 = 7.0147e-04
Validation auc = 0.757746
Epoch 215
Validation auc = 0.768310
Epoch 216
Validation auc = 0.781925
Epoch 217
Validation auc = 0.772770
Epoch 218
Validation auc = 0.763615
Epoch 219
Loss = 1.5260e+00, PNorm = 95.2428, GNorm = 71.4401, lr_0 = 6.9548e-04
Validation auc = 0.762441
Epoch 220
Validation auc = 0.767840
Epoch 221
Validation auc = 0.769718
Epoch 222
Validation auc = 0.773005
Epoch 223
Validation auc = 0.776056
Epoch 224
Loss = 3.1512e+00, PNorm = 95.5425, GNorm = 270.2169, lr_0 = 6.8953e-04
Validation auc = 0.775117
Epoch 225
Validation auc = 0.766432
Epoch 226
Validation auc = 0.741784
Epoch 227
Validation auc = 0.716667
Epoch 228
Validation auc = 0.686620
Epoch 229
Loss = 9.7627e-01, PNorm = 95.7846, GNorm = 62.1753, lr_0 = 6.8364e-04
Validation auc = 0.677230
Epoch 230
Validation auc = 0.684272
Epoch 231
Validation auc = 0.692254
Epoch 232
Validation auc = 0.693897
Epoch 233
Validation auc = 0.711033
Epoch 234
Loss = 1.2536e+00, PNorm = 96.0378, GNorm = 153.2072, lr_0 = 6.7779e-04
Validation auc = 0.718779
Epoch 235
Validation auc = 0.720892
Epoch 236
Validation auc = 0.714789
Epoch 237
Validation auc = 0.716197
Epoch 238
Validation auc = 0.713146
Epoch 239
Loss = 2.3830e+00, PNorm = 96.3374, GNorm = 150.6866, lr_0 = 6.7200e-04
Validation auc = 0.705399
Epoch 240
Validation auc = 0.705634
Epoch 241
Validation auc = 0.699765
Epoch 242
Validation auc = 0.710094
Epoch 243
Validation auc = 0.718545
Epoch 244
Loss = 1.6288e+00, PNorm = 96.5698, GNorm = 134.0194, lr_0 = 6.6625e-04
Validation auc = 0.731690
Epoch 245
Validation auc = 0.742958
Epoch 246
Validation auc = 0.745305
Epoch 247
Validation auc = 0.740141
Epoch 248
Validation auc = 0.715493
Epoch 249
Loss = 1.8740e+00, PNorm = 96.7788, GNorm = 238.2112, lr_0 = 6.6056e-04
Validation auc = 0.722770
Epoch 250
Validation auc = 0.735915
Epoch 251
Validation auc = 0.747418
Epoch 252
Validation auc = 0.742488
Epoch 253
Validation auc = 0.742723
Epoch 254
Loss = 2.0443e+00, PNorm = 97.1101, GNorm = 127.8177, lr_0 = 6.5491e-04
Validation auc = 0.747887
Epoch 255
Validation auc = 0.751643
Epoch 256
Validation auc = 0.760094
Epoch 257
Validation auc = 0.770423
Epoch 258
Validation auc = 0.770423
Epoch 259
Loss = 3.4635e+00, PNorm = 97.3876, GNorm = 380.7349, lr_0 = 6.4931e-04
Validation auc = 0.776761
Epoch 260
Validation auc = 0.772300
Epoch 261
Validation auc = 0.772770
Epoch 262
Validation auc = 0.762911
Epoch 263
Validation auc = 0.757746
Epoch 264
Loss = 9.1016e-01, PNorm = 97.6518, GNorm = 62.7445, lr_0 = 6.4376e-04
Validation auc = 0.759624
Epoch 265
Validation auc = 0.756103
Epoch 266
Validation auc = 0.766901
Epoch 267
Validation auc = 0.773239
Epoch 268
Validation auc = 0.776056
Epoch 269
Loss = 2.6699e+00, PNorm = 97.8423, GNorm = 202.4669, lr_0 = 6.3826e-04
Validation auc = 0.777230
Epoch 270
Validation auc = 0.776056
Epoch 271
Validation auc = 0.769718
Epoch 272
Validation auc = 0.761268
Epoch 273
Validation auc = 0.763615
Epoch 274
Loss = 1.0203e+00, PNorm = 98.1187, GNorm = 130.5691, lr_0 = 6.3280e-04
Validation auc = 0.770892
Epoch 275
Validation auc = 0.772300
Epoch 276
Validation auc = 0.765023
Epoch 277
Validation auc = 0.763615
Epoch 278
Validation auc = 0.762676
Epoch 279
Loss = 2.7724e+00, PNorm = 98.3876, GNorm = 135.8674, lr_0 = 6.2739e-04
Validation auc = 0.743897
Epoch 280
Validation auc = 0.748357
Epoch 281
Validation auc = 0.733099
Epoch 282
Validation auc = 0.719014
Epoch 283
Validation auc = 0.726526
Epoch 284
Loss = 9.1250e-01, PNorm = 98.6380, GNorm = 84.6008, lr_0 = 6.2203e-04
Validation auc = 0.723709
Epoch 285
Validation auc = 0.718779
Epoch 286
Validation auc = 0.730751
Epoch 287
Validation auc = 0.757746
Epoch 288
Validation auc = 0.773709
Epoch 289
Loss = 5.0014e-01, PNorm = 98.8519, GNorm = 52.9667, lr_0 = 6.1671e-04
Validation auc = 0.782394
Epoch 290
Validation auc = 0.782160
Epoch 291
Validation auc = 0.780282
Epoch 292
Validation auc = 0.774178
Epoch 293
Validation auc = 0.772300
Epoch 294
Loss = 2.1173e+00, PNorm = 99.1140, GNorm = 140.1664, lr_0 = 6.1144e-04
Validation auc = 0.763380
Epoch 295
Validation auc = 0.750000
Epoch 296
Validation auc = 0.744366
Epoch 297
Validation auc = 0.739202
Epoch 298
Validation auc = 0.734038
Epoch 299
Loss = 5.8759e-01, PNorm = 99.3663, GNorm = 110.6458, lr_0 = 6.0621e-04
Validation auc = 0.723239
Model 0 best validation auc = 0.792019 on epoch 212
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.777544
Ensemble test auc = 0.777544
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.623944
Epoch 1
Validation auc = 0.688263
Epoch 2
Validation auc = 0.711033
Epoch 3
Validation auc = 0.704225
Epoch 4
Loss = 3.0037e+01, PNorm = 78.6835, GNorm = 749.9091, lr_0 = 6.5000e-04
Validation auc = 0.701408
Epoch 5
Validation auc = 0.697183
Epoch 6
Validation auc = 0.728169
Epoch 7
Validation auc = 0.763615
Epoch 8
Validation auc = 0.618779
Epoch 9
Loss = 5.6802e+01, PNorm = 78.9937, GNorm = 2090.1098, lr_0 = 9.9743e-04
Validation auc = 0.740845
Epoch 10
Validation auc = 0.702817
Epoch 11
Validation auc = 0.702582
Epoch 12
Validation auc = 0.671831
Epoch 13
Validation auc = 0.688263
Epoch 14
Loss = 4.6367e+01, PNorm = 79.2991, GNorm = 1105.7555, lr_0 = 9.8890e-04
Validation auc = 0.635211
Epoch 15
Validation auc = 0.701878
Epoch 16
Validation auc = 0.626526
Epoch 17
Validation auc = 0.645540
Epoch 18
Validation auc = 0.711033
Epoch 19
Loss = 9.0332e+01, PNorm = 79.5247, GNorm = 1824.0283, lr_0 = 9.8045e-04
Validation auc = 0.711033
Epoch 20
Validation auc = 0.706338
Epoch 21
Validation auc = 0.599061
Epoch 22
Validation auc = 0.706338
Epoch 23
Validation auc = 0.730516
Epoch 24
Loss = 4.6620e+01, PNorm = 79.6713, GNorm = 991.5363, lr_0 = 9.7207e-04
Validation auc = 0.732629
Epoch 25
Validation auc = 0.668545
Epoch 26
Validation auc = 0.691549
Epoch 27
Validation auc = 0.707746
Epoch 28
Validation auc = 0.717840
Epoch 29
Loss = 3.5143e+01, PNorm = 79.7845, GNorm = 550.1731, lr_0 = 9.6376e-04
Validation auc = 0.713850
Epoch 30
Validation auc = 0.706808
Epoch 31
Validation auc = 0.750704
Epoch 32
Validation auc = 0.751878
Epoch 33
Validation auc = 0.740610
Epoch 34
Loss = 3.9756e+01, PNorm = 79.8834, GNorm = 795.6629, lr_0 = 9.5552e-04
Validation auc = 0.649765
Epoch 35
Validation auc = 0.682629
Epoch 36
Validation auc = 0.738732
Epoch 37
Validation auc = 0.734038
Epoch 38
Validation auc = 0.698826
Epoch 39
Loss = 2.3685e+01, PNorm = 80.0094, GNorm = 526.0860, lr_0 = 9.4735e-04
Validation auc = 0.688967
Epoch 40
Validation auc = 0.714319
Epoch 41
Validation auc = 0.708216
Epoch 42
Validation auc = 0.734507
Epoch 43
Validation auc = 0.724883
Epoch 44
Loss = 2.3324e+01, PNorm = 80.1409, GNorm = 620.4603, lr_0 = 9.3925e-04
Validation auc = 0.666197
Epoch 45
Validation auc = 0.675587
Epoch 46
Validation auc = 0.742019
Epoch 47
Validation auc = 0.742723
Epoch 48
Validation auc = 0.676056
Epoch 49
Loss = 2.2785e+01, PNorm = 80.2965, GNorm = 641.7736, lr_0 = 9.3122e-04
Validation auc = 0.623005
Epoch 50
Validation auc = 0.676291
Epoch 51
Validation auc = 0.718779
Epoch 52
Validation auc = 0.696479
Epoch 53
Validation auc = 0.620188
Epoch 54
Loss = 2.9367e+01, PNorm = 80.5679, GNorm = 738.7865, lr_0 = 9.2326e-04
Validation auc = 0.621831
Epoch 55
Validation auc = 0.701408
Epoch 56
Validation auc = 0.688498
Epoch 57
Validation auc = 0.622066
Epoch 58
Validation auc = 0.633333
Epoch 59
Loss = 2.2548e+01, PNorm = 80.7998, GNorm = 957.4242, lr_0 = 9.1537e-04
Validation auc = 0.667606
Epoch 60
Validation auc = 0.665728
Epoch 61
Validation auc = 0.677934
Epoch 62
Validation auc = 0.686854
Epoch 63
Validation auc = 0.670892
Epoch 64
Loss = 3.3400e+01, PNorm = 81.0674, GNorm = 1142.0288, lr_0 = 9.0754e-04
Validation auc = 0.686854
Epoch 65
Validation auc = 0.724883
Epoch 66
Validation auc = 0.691549
Epoch 67
Validation auc = 0.684977
Epoch 68
Validation auc = 0.710329
Epoch 69
Loss = 1.7701e+01, PNorm = 81.5091, GNorm = 687.2673, lr_0 = 8.9978e-04
Validation auc = 0.709390
Epoch 70
Validation auc = 0.697653
Epoch 71
Validation auc = 0.712207
Epoch 72
Validation auc = 0.754460
Epoch 73
Validation auc = 0.736620
Epoch 74
Loss = 1.4216e+01, PNorm = 82.0665, GNorm = 461.9628, lr_0 = 8.9209e-04
Validation auc = 0.709390
Epoch 75
Validation auc = 0.672300
Epoch 76
Validation auc = 0.711737
Epoch 77
Validation auc = 0.715023
Epoch 78
Validation auc = 0.720188
Epoch 79
Loss = 1.3986e+01, PNorm = 82.5301, GNorm = 626.4590, lr_0 = 8.8447e-04
Validation auc = 0.711268
Epoch 80
Validation auc = 0.686385
Epoch 81
Validation auc = 0.688028
Epoch 82
Validation auc = 0.693427
Epoch 83
Validation auc = 0.728638
Epoch 84
Loss = 1.3255e+01, PNorm = 83.1703, GNorm = 760.9386, lr_0 = 8.7691e-04
Validation auc = 0.711268
Epoch 85
Validation auc = 0.673005
Epoch 86
Validation auc = 0.663615
Epoch 87
Validation auc = 0.730516
Epoch 88
Validation auc = 0.721596
Epoch 89
Loss = 1.2239e+01, PNorm = 83.5735, GNorm = 324.2778, lr_0 = 8.6941e-04
Validation auc = 0.678404
Epoch 90
Validation auc = 0.675352
Epoch 91
Validation auc = 0.708451
Epoch 92
Validation auc = 0.716667
Epoch 93
Validation auc = 0.712911
Epoch 94
Loss = 8.9796e+00, PNorm = 84.0688, GNorm = 483.4552, lr_0 = 8.6198e-04
Validation auc = 0.668779
Epoch 95
Validation auc = 0.700235
Epoch 96
Validation auc = 0.734272
Epoch 97
Validation auc = 0.741784
Epoch 98
Validation auc = 0.714319
Epoch 99
Loss = 1.1173e+01, PNorm = 84.5489, GNorm = 682.5373, lr_0 = 8.5461e-04
Validation auc = 0.688028
Epoch 100
Validation auc = 0.742019
Epoch 101
Validation auc = 0.740376
Epoch 102
Validation auc = 0.731690
Epoch 103
Validation auc = 0.719953
Epoch 104
Loss = 3.1906e+00, PNorm = 85.0816, GNorm = 290.2277, lr_0 = 8.4730e-04
Validation auc = 0.709859
Epoch 105
Validation auc = 0.705869
Epoch 106
Validation auc = 0.723709
Epoch 107
Validation auc = 0.734272
Epoch 108
Validation auc = 0.734272
Epoch 109
Loss = 1.0627e+01, PNorm = 85.7611, GNorm = 913.0205, lr_0 = 8.4006e-04
Validation auc = 0.757277
Epoch 110
Validation auc = 0.707042
Epoch 111
Validation auc = 0.722535
Epoch 112
Validation auc = 0.737559
Epoch 113
Validation auc = 0.752817
Epoch 114
Loss = 7.2020e+00, PNorm = 86.1637, GNorm = 330.0360, lr_0 = 8.3288e-04
Validation auc = 0.711502
Epoch 115
Validation auc = 0.734977
Epoch 116
Validation auc = 0.726761
Epoch 117
Validation auc = 0.720188
Epoch 118
Validation auc = 0.740845
Epoch 119
Loss = 9.1008e+00, PNorm = 86.7304, GNorm = 387.6884, lr_0 = 8.2576e-04
Validation auc = 0.750000
Epoch 120
Validation auc = 0.750469
Epoch 121
Validation auc = 0.774413
Epoch 122
Validation auc = 0.769718
Epoch 123
Validation auc = 0.768545
Epoch 124
Loss = 9.3155e+00, PNorm = 87.3353, GNorm = 306.3676, lr_0 = 8.1870e-04
Validation auc = 0.736620
Epoch 125
Validation auc = 0.759155
Epoch 126
Validation auc = 0.779108
Epoch 127
Validation auc = 0.769014
Epoch 128
Validation auc = 0.794366
Epoch 129
Loss = 1.1156e+01, PNorm = 87.8442, GNorm = 897.4318, lr_0 = 8.1170e-04
Validation auc = 0.793897
Epoch 130
Validation auc = 0.763146
Epoch 131
Validation auc = 0.783333
Epoch 132
Validation auc = 0.790141
Epoch 133
Validation auc = 0.786620
Epoch 134
Loss = 6.0763e+00, PNorm = 88.3251, GNorm = 480.8641, lr_0 = 8.0476e-04
Validation auc = 0.750469
Epoch 135
Validation auc = 0.726291
Epoch 136
Validation auc = 0.741549
Epoch 137
Validation auc = 0.767606
Epoch 138
Validation auc = 0.783333
Epoch 139
Loss = 4.5071e+00, PNorm = 88.7376, GNorm = 333.0754, lr_0 = 7.9788e-04
Validation auc = 0.784272
Epoch 140
Validation auc = 0.758216
Epoch 141
Validation auc = 0.777230
Epoch 142
Validation auc = 0.796009
Epoch 143
Validation auc = 0.785446
Epoch 144
Loss = 7.5675e+00, PNorm = 89.0985, GNorm = 376.2028, lr_0 = 7.9106e-04
Validation auc = 0.793662
Epoch 145
Validation auc = 0.784977
Epoch 146
Validation auc = 0.773474
Epoch 147
Validation auc = 0.753991
Epoch 148
Validation auc = 0.758920
Epoch 149
Loss = 6.8002e+00, PNorm = 89.4313, GNorm = 294.5714, lr_0 = 7.8430e-04
Validation auc = 0.771362
Epoch 150
Validation auc = 0.755869
Epoch 151
Validation auc = 0.732864
Epoch 152
Validation auc = 0.732629
Epoch 153
Validation auc = 0.753756
Epoch 154
Loss = 4.8052e+00, PNorm = 89.8489, GNorm = 496.4745, lr_0 = 7.7759e-04
Validation auc = 0.761737
Epoch 155
Validation auc = 0.749531
Epoch 156
Validation auc = 0.724648
Epoch 157
Validation auc = 0.719249
Epoch 158
Validation auc = 0.732629
Epoch 159
Loss = 5.6247e+00, PNorm = 90.2935, GNorm = 542.3862, lr_0 = 7.7095e-04
Validation auc = 0.751643
Epoch 160
Validation auc = 0.750939
Epoch 161
Validation auc = 0.764789
Epoch 162
Validation auc = 0.777700
Epoch 163
Validation auc = 0.753052
Epoch 164
Loss = 4.8902e+00, PNorm = 90.8329, GNorm = 181.1465, lr_0 = 7.6436e-04
Validation auc = 0.733803
Epoch 165
Validation auc = 0.757512
Epoch 166
Validation auc = 0.779812
Epoch 167
Validation auc = 0.773709
Epoch 168
Validation auc = 0.757277
Epoch 169
Loss = 2.1314e+00, PNorm = 91.3015, GNorm = 75.0021, lr_0 = 7.5782e-04
Validation auc = 0.720657
Epoch 170
Validation auc = 0.715493
Epoch 171
Validation auc = 0.769953
Epoch 172
Validation auc = 0.762676
Epoch 173
Validation auc = 0.767371
Epoch 174
Loss = 3.8486e+00, PNorm = 91.6734, GNorm = 230.2364, lr_0 = 7.5134e-04
Validation auc = 0.775352
Epoch 175
Validation auc = 0.753286
Epoch 176
Validation auc = 0.750000
Epoch 177
Validation auc = 0.761737
Epoch 178
Validation auc = 0.763615
Epoch 179
Loss = 4.0283e+00, PNorm = 92.0715, GNorm = 214.0389, lr_0 = 7.4492e-04
Validation auc = 0.763380
Epoch 180
Validation auc = 0.765962
Epoch 181
Validation auc = 0.761033
Epoch 182
Validation auc = 0.750469
Epoch 183
Validation auc = 0.714789
Epoch 184
Loss = 4.4420e+00, PNorm = 92.4242, GNorm = 148.8582, lr_0 = 7.3855e-04
Validation auc = 0.704695
Epoch 185
Validation auc = 0.722300
Epoch 186
Validation auc = 0.728169
Epoch 187
Validation auc = 0.729577
Epoch 188
Validation auc = 0.725587
Epoch 189
Loss = 1.5458e+00, PNorm = 92.7739, GNorm = 269.6154, lr_0 = 7.3224e-04
Validation auc = 0.726056
Epoch 190
Validation auc = 0.722300
Epoch 191
Validation auc = 0.736150
Epoch 192
Validation auc = 0.748357
Epoch 193
Validation auc = 0.745305
Epoch 194
Loss = 3.1817e+00, PNorm = 93.0698, GNorm = 113.1790, lr_0 = 7.2598e-04
Validation auc = 0.746009
Epoch 195
Validation auc = 0.752347
Epoch 196
Validation auc = 0.762676
Epoch 197
Validation auc = 0.768075
Epoch 198
Validation auc = 0.763850
Epoch 199
Loss = 3.0379e+00, PNorm = 93.4347, GNorm = 220.6087, lr_0 = 7.1977e-04
Validation auc = 0.757746
Epoch 200
Validation auc = 0.748826
Epoch 201
Validation auc = 0.752347
Epoch 202
Validation auc = 0.751408
Epoch 203
Validation auc = 0.745775
Epoch 204
Loss = 1.9623e+00, PNorm = 93.8998, GNorm = 88.5990, lr_0 = 7.1362e-04
Validation auc = 0.754460
Epoch 205
Validation auc = 0.749296
Epoch 206
Validation auc = 0.749765
Epoch 207
Validation auc = 0.740376
Epoch 208
Validation auc = 0.727465
Epoch 209
Loss = 1.2401e+00, PNorm = 94.2688, GNorm = 118.1211, lr_0 = 7.0752e-04
Validation auc = 0.731690
Epoch 210
Validation auc = 0.729108
Epoch 211
Validation auc = 0.732160
Epoch 212
Validation auc = 0.733803
Epoch 213
Validation auc = 0.749765
Epoch 214
Loss = 4.6471e+00, PNorm = 94.6499, GNorm = 253.4871, lr_0 = 7.0147e-04
Validation auc = 0.773005
Epoch 215
Validation auc = 0.764789
Epoch 216
Validation auc = 0.771362
Epoch 217
Validation auc = 0.776761
Epoch 218
Validation auc = 0.769718
Epoch 219
Loss = 2.7958e+00, PNorm = 95.0177, GNorm = 252.5292, lr_0 = 6.9548e-04
Validation auc = 0.770892
Epoch 220
Validation auc = 0.753052
Epoch 221
Validation auc = 0.740376
Epoch 222
Validation auc = 0.746009
Epoch 223
Validation auc = 0.759155
Epoch 224
Loss = 5.4379e-01, PNorm = 95.3696, GNorm = 23.6744, lr_0 = 6.8953e-04
Validation auc = 0.768310
Epoch 225
Validation auc = 0.771831
Epoch 226
Validation auc = 0.769249
Epoch 227
Validation auc = 0.763615
Epoch 228
Validation auc = 0.753286
Epoch 229
Loss = 1.7192e+00, PNorm = 95.6758, GNorm = 145.0298, lr_0 = 6.8364e-04
Validation auc = 0.745775
Epoch 230
Validation auc = 0.747183
Epoch 231
Validation auc = 0.755399
Epoch 232
Validation auc = 0.744366
Epoch 233
Validation auc = 0.736385
Epoch 234
Loss = 3.5679e+00, PNorm = 95.9291, GNorm = 374.8124, lr_0 = 6.7779e-04
Validation auc = 0.742254
Epoch 235
Validation auc = 0.749061
Epoch 236
Validation auc = 0.758216
Epoch 237
Validation auc = 0.746714
Epoch 238
Validation auc = 0.744131
Epoch 239
Loss = 1.8373e+00, PNorm = 96.2403, GNorm = 143.5748, lr_0 = 6.7200e-04
Validation auc = 0.760329
Epoch 240
Validation auc = 0.769484
Epoch 241
Validation auc = 0.766667
Epoch 242
Validation auc = 0.756338
Epoch 243
Validation auc = 0.744366
Epoch 244
Loss = 9.9385e-01, PNorm = 96.5543, GNorm = 118.9062, lr_0 = 6.6625e-04
Validation auc = 0.727934
Epoch 245
Validation auc = 0.723709
Epoch 246
Validation auc = 0.724883
Epoch 247
Validation auc = 0.724648
Epoch 248
Validation auc = 0.721362
Epoch 249
Loss = 3.1399e-01, PNorm = 96.7729, GNorm = 81.1129, lr_0 = 6.6056e-04
Validation auc = 0.718545
Epoch 250
Validation auc = 0.714554
Epoch 251
Validation auc = 0.715962
Epoch 252
Validation auc = 0.725822
Epoch 253
Validation auc = 0.728169
Epoch 254
Loss = 1.4839e+00, PNorm = 97.0318, GNorm = 81.1978, lr_0 = 6.5491e-04
Validation auc = 0.726995
Epoch 255
Validation auc = 0.720423
Epoch 256
Validation auc = 0.728638
Epoch 257
Validation auc = 0.739906
Epoch 258
Validation auc = 0.744836
Epoch 259
Loss = 1.5628e+00, PNorm = 97.3055, GNorm = 211.8318, lr_0 = 6.4931e-04
Validation auc = 0.743192
Epoch 260
Validation auc = 0.742723
Epoch 261
Validation auc = 0.737324
Epoch 262
Validation auc = 0.733333
Epoch 263
Validation auc = 0.739202
Epoch 264
Loss = 4.5428e-01, PNorm = 97.5985, GNorm = 45.5712, lr_0 = 6.4376e-04
Validation auc = 0.739906
Epoch 265
Validation auc = 0.736150
Epoch 266
Validation auc = 0.729577
Epoch 267
Validation auc = 0.726526
Epoch 268
Validation auc = 0.725352
Epoch 269
Loss = 8.9463e-01, PNorm = 97.8964, GNorm = 176.5774, lr_0 = 6.3826e-04
Validation auc = 0.719953
Epoch 270
Validation auc = 0.715258
Epoch 271
Validation auc = 0.712207
Epoch 272
Validation auc = 0.711972
Epoch 273
Validation auc = 0.713380
Epoch 274
Loss = 1.0163e+00, PNorm = 98.1703, GNorm = 177.6565, lr_0 = 6.3280e-04
Validation auc = 0.708685
Epoch 275
Validation auc = 0.721127
Epoch 276
Validation auc = 0.742958
Epoch 277
Validation auc = 0.757981
Epoch 278
Validation auc = 0.758216
Epoch 279
Loss = 8.6499e-01, PNorm = 98.4800, GNorm = 84.9534, lr_0 = 6.2739e-04
Validation auc = 0.764554
Epoch 280
Validation auc = 0.772066
Epoch 281
Validation auc = 0.768545
Epoch 282
Validation auc = 0.767606
Epoch 283
Validation auc = 0.769953
Epoch 284
Loss = 8.7034e-01, PNorm = 98.7387, GNorm = 150.2793, lr_0 = 6.2203e-04
Validation auc = 0.760798
Epoch 285
Validation auc = 0.747183
Epoch 286
Validation auc = 0.740845
Epoch 287
Validation auc = 0.739906
Epoch 288
Validation auc = 0.737559
Epoch 289
Loss = 9.4665e-01, PNorm = 98.9975, GNorm = 159.3290, lr_0 = 6.1671e-04
Validation auc = 0.744131
Epoch 290
Validation auc = 0.746009
Epoch 291
Validation auc = 0.742488
Epoch 292
Validation auc = 0.745305
Epoch 293
Validation auc = 0.747183
Epoch 294
Loss = 1.7416e+00, PNorm = 99.2716, GNorm = 101.7474, lr_0 = 6.1144e-04
Validation auc = 0.747653
Epoch 295
Validation auc = 0.747418
Epoch 296
Validation auc = 0.747887
Epoch 297
Validation auc = 0.744131
Epoch 298
Validation auc = 0.741784
Epoch 299
Loss = 1.1942e+00, PNorm = 99.5468, GNorm = 181.0628, lr_0 = 6.0621e-04
Validation auc = 0.740376
Model 0 best validation auc = 0.796009 on epoch 142
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.743227
Ensemble test auc = 0.743227
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.535681
Epoch 1
Validation auc = 0.695305
Epoch 2
Validation auc = 0.731925
Epoch 3
Validation auc = 0.703991
Epoch 4
Loss = 6.3389e+01, PNorm = 78.6853, GNorm = 1766.1614, lr_0 = 6.5000e-04
Validation auc = 0.690845
Epoch 5
Validation auc = 0.741315
Epoch 6
Validation auc = 0.581690
Epoch 7
Validation auc = 0.739202
Epoch 8
Validation auc = 0.726291
Epoch 9
Loss = 4.8461e+01, PNorm = 78.9715, GNorm = 718.0269, lr_0 = 9.9743e-04
Validation auc = 0.734742
Epoch 10
Validation auc = 0.678873
Epoch 11
Validation auc = 0.692958
Epoch 12
Validation auc = 0.708920
Epoch 13
Validation auc = 0.711972
Epoch 14
Loss = 4.0966e+01, PNorm = 79.2855, GNorm = 1621.6612, lr_0 = 9.8890e-04
Validation auc = 0.641315
Epoch 15
Validation auc = 0.698122
Epoch 16
Validation auc = 0.692488
Epoch 17
Validation auc = 0.720892
Epoch 18
Validation auc = 0.642019
Epoch 19
Loss = 3.5853e+01, PNorm = 79.4916, GNorm = 432.7320, lr_0 = 9.8045e-04
Validation auc = 0.669484
Epoch 20
Validation auc = 0.732160
Epoch 21
Validation auc = 0.714085
Epoch 22
Validation auc = 0.644601
Epoch 23
Validation auc = 0.744601
Epoch 24
Loss = 3.2013e+01, PNorm = 79.6121, GNorm = 1145.4059, lr_0 = 9.7207e-04
Validation auc = 0.742958
Epoch 25
Validation auc = 0.708685
Epoch 26
Validation auc = 0.657277
Epoch 27
Validation auc = 0.710094
Epoch 28
Validation auc = 0.757746
Epoch 29
Loss = 4.1616e+01, PNorm = 79.7025, GNorm = 642.4764, lr_0 = 9.6376e-04
Validation auc = 0.739437
Epoch 30
Validation auc = 0.649296
Epoch 31
Validation auc = 0.680751
Epoch 32
Validation auc = 0.746479
Epoch 33
Validation auc = 0.752582
Epoch 34
Loss = 2.8530e+01, PNorm = 79.7951, GNorm = 502.4284, lr_0 = 9.5552e-04
Validation auc = 0.712911
Epoch 35
Validation auc = 0.719249
Epoch 36
Validation auc = 0.720423
Epoch 37
Validation auc = 0.713380
Epoch 38
Validation auc = 0.691549
Epoch 39
Loss = 2.3867e+01, PNorm = 79.9253, GNorm = 589.9500, lr_0 = 9.4735e-04
Validation auc = 0.672770
Epoch 40
Validation auc = 0.674178
Epoch 41
Validation auc = 0.651878
Epoch 42
Validation auc = 0.702817
Epoch 43
Validation auc = 0.677934
Epoch 44
Loss = 3.4363e+01, PNorm = 80.0874, GNorm = 666.8751, lr_0 = 9.3925e-04
Validation auc = 0.695070
Epoch 45
Validation auc = 0.685681
Epoch 46
Validation auc = 0.688028
Epoch 47
Validation auc = 0.652817
Epoch 48
Validation auc = 0.636620
Epoch 49
Loss = 3.9233e+01, PNorm = 80.3304, GNorm = 1011.4807, lr_0 = 9.3122e-04
Validation auc = 0.709624
Epoch 50
Validation auc = 0.756808
Epoch 51
Validation auc = 0.682160
Epoch 52
Validation auc = 0.609155
Epoch 53
Validation auc = 0.729108
Epoch 54
Loss = 4.0388e+01, PNorm = 80.5540, GNorm = 988.8135, lr_0 = 9.2326e-04
Validation auc = 0.688263
Epoch 55
Validation auc = 0.690141
Epoch 56
Validation auc = 0.548122
Epoch 57
Validation auc = 0.692723
Epoch 58
Validation auc = 0.698592
Epoch 59
Loss = 3.8552e+01, PNorm = 80.8275, GNorm = 743.6314, lr_0 = 9.1537e-04
Validation auc = 0.728404
Epoch 60
Validation auc = 0.692723
Epoch 61
Validation auc = 0.716432
Epoch 62
Validation auc = 0.709390
Epoch 63
Validation auc = 0.721127
Epoch 64
Loss = 1.4681e+01, PNorm = 81.1961, GNorm = 614.5460, lr_0 = 9.0754e-04
Validation auc = 0.734742
Epoch 65
Validation auc = 0.717840
Epoch 66
Validation auc = 0.728404
Epoch 67
Validation auc = 0.735211
Epoch 68
Validation auc = 0.732394
Epoch 69
Loss = 1.4883e+01, PNorm = 81.5934, GNorm = 724.4955, lr_0 = 8.9978e-04
Validation auc = 0.738028
Epoch 70
Validation auc = 0.738498
Epoch 71
Validation auc = 0.750235
Epoch 72
Validation auc = 0.732160
Epoch 73
Validation auc = 0.730986
Epoch 74
Loss = 1.5337e+01, PNorm = 81.9990, GNorm = 361.2234, lr_0 = 8.9209e-04
Validation auc = 0.727465
Epoch 75
Validation auc = 0.733333
Epoch 76
Validation auc = 0.714319
Epoch 77
Validation auc = 0.719249
Epoch 78
Validation auc = 0.719953
Epoch 79
Loss = 1.0698e+01, PNorm = 82.3477, GNorm = 326.7238, lr_0 = 8.8447e-04
Validation auc = 0.706103
Epoch 80
Validation auc = 0.719484
Epoch 81
Validation auc = 0.715728
Epoch 82
Validation auc = 0.722066
Epoch 83
Validation auc = 0.737793
Epoch 84
Loss = 2.1365e+01, PNorm = 82.7919, GNorm = 290.7351, lr_0 = 8.7691e-04
Validation auc = 0.746009
Epoch 85
Validation auc = 0.733333
Epoch 86
Validation auc = 0.692254
Epoch 87
Validation auc = 0.738732
Epoch 88
Validation auc = 0.720657
Epoch 89
Loss = 1.6817e+01, PNorm = 83.1102, GNorm = 455.8554, lr_0 = 8.6941e-04
Validation auc = 0.746009
Epoch 90
Validation auc = 0.693897
Epoch 91
Validation auc = 0.686854
Epoch 92
Validation auc = 0.746948
Epoch 93
Validation auc = 0.752113
Epoch 94
Loss = 1.3784e+01, PNorm = 83.5403, GNorm = 488.4780, lr_0 = 8.6198e-04
Validation auc = 0.748592
Epoch 95
Validation auc = 0.723474
Epoch 96
Validation auc = 0.687324
Epoch 97
Validation auc = 0.723709
Epoch 98
Validation auc = 0.750235
Epoch 99
Loss = 1.3675e+01, PNorm = 83.9409, GNorm = 503.1571, lr_0 = 8.5461e-04
Validation auc = 0.749531
Epoch 100
Validation auc = 0.741315
Epoch 101
Validation auc = 0.697183
Epoch 102
Validation auc = 0.667606
Epoch 103
Validation auc = 0.671362
Epoch 104
Loss = 1.5482e+01, PNorm = 84.3280, GNorm = 366.3067, lr_0 = 8.4730e-04
Validation auc = 0.697183
Epoch 105
Validation auc = 0.718310
Epoch 106
Validation auc = 0.720188
Epoch 107
Validation auc = 0.712207
Epoch 108
Validation auc = 0.706103
Epoch 109
Loss = 2.4543e+00, PNorm = 84.7022, GNorm = 268.7997, lr_0 = 8.4006e-04
Validation auc = 0.683099
Epoch 110
Validation auc = 0.687559
Epoch 111
Validation auc = 0.701643
Epoch 112
Validation auc = 0.728169
Epoch 113
Validation auc = 0.729108
Epoch 114
Loss = 1.2369e+01, PNorm = 85.0649, GNorm = 458.0594, lr_0 = 8.3288e-04
Validation auc = 0.682629
Epoch 115
Validation auc = 0.669718
Epoch 116
Validation auc = 0.717606
Epoch 117
Validation auc = 0.722770
Epoch 118
Validation auc = 0.700469
Epoch 119
Loss = 9.0190e+00, PNorm = 85.3678, GNorm = 563.3335, lr_0 = 8.2576e-04
Validation auc = 0.724648
Epoch 120
Validation auc = 0.670188
Epoch 121
Validation auc = 0.678873
Epoch 122
Validation auc = 0.719484
Epoch 123
Validation auc = 0.738028
Epoch 124
Loss = 8.1456e+00, PNorm = 85.7242, GNorm = 487.6845, lr_0 = 8.1870e-04
Validation auc = 0.726056
Epoch 125
Validation auc = 0.755869
Epoch 126
Validation auc = 0.723709
Epoch 127
Validation auc = 0.723944
Epoch 128
Validation auc = 0.742019
Epoch 129
Loss = 8.8540e+00, PNorm = 86.1681, GNorm = 203.5319, lr_0 = 8.1170e-04
Validation auc = 0.711737
Epoch 130
Validation auc = 0.704930
Epoch 131
Validation auc = 0.738967
Epoch 132
Validation auc = 0.745775
Epoch 133
Validation auc = 0.749531
Epoch 134
Loss = 5.7767e+00, PNorm = 86.5030, GNorm = 237.6436, lr_0 = 8.0476e-04
Validation auc = 0.766197
Epoch 135
Validation auc = 0.764319
Epoch 136
Validation auc = 0.772066
Epoch 137
Validation auc = 0.775822
Epoch 138
Validation auc = 0.751643
Epoch 139
Loss = 5.0755e+00, PNorm = 86.8660, GNorm = 167.9798, lr_0 = 7.9788e-04
Validation auc = 0.724648
Epoch 140
Validation auc = 0.714554
Epoch 141
Validation auc = 0.728638
Epoch 142
Validation auc = 0.732394
Epoch 143
Validation auc = 0.725587
Epoch 144
Loss = 2.9970e+00, PNorm = 87.2172, GNorm = 242.0467, lr_0 = 7.9106e-04
Validation auc = 0.711268
Epoch 145
Validation auc = 0.713850
Epoch 146
Validation auc = 0.725822
Epoch 147
Validation auc = 0.734507
Epoch 148
Validation auc = 0.741549
Epoch 149
Loss = 4.3509e+00, PNorm = 87.5211, GNorm = 224.3810, lr_0 = 7.8430e-04
Validation auc = 0.732864
Epoch 150
Validation auc = 0.721127
Epoch 151
Validation auc = 0.713380
Epoch 152
Validation auc = 0.715258
Epoch 153
Validation auc = 0.726291
Epoch 154
Loss = 3.5279e+00, PNorm = 87.8259, GNorm = 201.8441, lr_0 = 7.7759e-04
Validation auc = 0.720657
Epoch 155
Validation auc = 0.728169
Epoch 156
Validation auc = 0.721362
Epoch 157
Validation auc = 0.698122
Epoch 158
Validation auc = 0.703521
Epoch 159
Loss = 3.9383e+00, PNorm = 88.1569, GNorm = 242.5922, lr_0 = 7.7095e-04
Validation auc = 0.719014
Epoch 160
Validation auc = 0.730282
Epoch 161
Validation auc = 0.735211
Epoch 162
Validation auc = 0.734038
Epoch 163
Validation auc = 0.717606
Epoch 164
Loss = 6.1430e+00, PNorm = 88.6218, GNorm = 394.5249, lr_0 = 7.6436e-04
Validation auc = 0.709624
Epoch 165
Validation auc = 0.728638
Epoch 166
Validation auc = 0.742254
Epoch 167
Validation auc = 0.750469
Epoch 168
Validation auc = 0.753756
Epoch 169
Loss = 5.3689e+00, PNorm = 89.1830, GNorm = 241.7099, lr_0 = 7.5782e-04
Validation auc = 0.736150
Epoch 170
Validation auc = 0.716901
Epoch 171
Validation auc = 0.727230
Epoch 172
Validation auc = 0.741315
Epoch 173
Validation auc = 0.733099
Epoch 174
Loss = 2.6166e+00, PNorm = 89.5487, GNorm = 219.6517, lr_0 = 7.5134e-04
Validation auc = 0.728873
Epoch 175
Validation auc = 0.716667
Epoch 176
Validation auc = 0.686854
Epoch 177
Validation auc = 0.672300
Epoch 178
Validation auc = 0.692488
Epoch 179
Loss = 2.6224e+00, PNorm = 89.8265, GNorm = 190.9835, lr_0 = 7.4492e-04
Validation auc = 0.699296
Epoch 180
Validation auc = 0.680047
Epoch 181
Validation auc = 0.697418
Epoch 182
Validation auc = 0.696948
Epoch 183
Validation auc = 0.678169
Epoch 184
Loss = 5.7187e+00, PNorm = 90.1391, GNorm = 525.1322, lr_0 = 7.3855e-04
Validation auc = 0.688732
Epoch 185
Validation auc = 0.715962
Epoch 186
Validation auc = 0.735446
Epoch 187
Validation auc = 0.751174
Epoch 188
Validation auc = 0.758685
Epoch 189
Loss = 2.7243e+00, PNorm = 90.3903, GNorm = 331.5222, lr_0 = 7.3224e-04
Validation auc = 0.764789
Epoch 190
Validation auc = 0.759390
Epoch 191
Validation auc = 0.739671
Epoch 192
Validation auc = 0.722535
Epoch 193
Validation auc = 0.722300
Epoch 194
Loss = 2.3023e+00, PNorm = 90.7253, GNorm = 112.4401, lr_0 = 7.2598e-04
Validation auc = 0.735681
Epoch 195
Validation auc = 0.736150
Epoch 196
Validation auc = 0.740141
Epoch 197
Validation auc = 0.733333
Epoch 198
Validation auc = 0.733333
Epoch 199
Loss = 5.5364e+00, PNorm = 90.9771, GNorm = 409.9387, lr_0 = 7.1977e-04
Validation auc = 0.734272
Epoch 200
Validation auc = 0.738498
Epoch 201
Validation auc = 0.739202
Epoch 202
Validation auc = 0.731221
Epoch 203
Validation auc = 0.702582
Epoch 204
Loss = 2.8360e+00, PNorm = 91.3172, GNorm = 135.4909, lr_0 = 7.1362e-04
Validation auc = 0.696009
Epoch 205
Validation auc = 0.687793
Epoch 206
Validation auc = 0.700469
Epoch 207
Validation auc = 0.698357
Epoch 208
Validation auc = 0.695775
Epoch 209
Loss = 2.6118e+00, PNorm = 91.6103, GNorm = 119.7720, lr_0 = 7.0752e-04
Validation auc = 0.710798
Epoch 210
Validation auc = 0.715493
Epoch 211
Validation auc = 0.713850
Epoch 212
Validation auc = 0.710798
Epoch 213
Validation auc = 0.716432
Epoch 214
Loss = 5.2054e+00, PNorm = 91.9040, GNorm = 257.8266, lr_0 = 7.0147e-04
Validation auc = 0.727465
Epoch 215
Validation auc = 0.737324
Epoch 216
Validation auc = 0.744601
Epoch 217
Validation auc = 0.734742
Epoch 218
Validation auc = 0.743427
Epoch 219
Loss = 2.0404e+00, PNorm = 92.2786, GNorm = 113.4818, lr_0 = 6.9548e-04
Validation auc = 0.755164
Epoch 220
Validation auc = 0.752347
Epoch 221
Validation auc = 0.744366
Epoch 222
Validation auc = 0.756103
Epoch 223
Validation auc = 0.765728
Epoch 224
Loss = 2.7829e+00, PNorm = 92.6052, GNorm = 241.5466, lr_0 = 6.8953e-04
Validation auc = 0.766901
Epoch 225
Validation auc = 0.755869
Epoch 226
Validation auc = 0.750000
Epoch 227
Validation auc = 0.751174
Epoch 228
Validation auc = 0.754695
Epoch 229
Loss = 1.1327e+00, PNorm = 92.8560, GNorm = 95.1106, lr_0 = 6.8364e-04
Validation auc = 0.754460
Epoch 230
Validation auc = 0.735211
Epoch 231
Validation auc = 0.732394
Epoch 232
Validation auc = 0.727230
Epoch 233
Validation auc = 0.719718
Epoch 234
Loss = 8.4401e-01, PNorm = 93.1104, GNorm = 94.9797, lr_0 = 6.7779e-04
Validation auc = 0.716901
Epoch 235
Validation auc = 0.720423
Epoch 236
Validation auc = 0.729577
Epoch 237
Validation auc = 0.735681
Epoch 238
Validation auc = 0.743192
Epoch 239
Loss = 2.2430e+00, PNorm = 93.4324, GNorm = 149.6256, lr_0 = 6.7200e-04
Validation auc = 0.755869
Epoch 240
Validation auc = 0.763850
Epoch 241
Validation auc = 0.766197
Epoch 242
Validation auc = 0.761502
Epoch 243
Validation auc = 0.753052
Epoch 244
Loss = 3.6687e+00, PNorm = 93.7323, GNorm = 309.2575, lr_0 = 6.6625e-04
Validation auc = 0.755634
Epoch 245
Validation auc = 0.753991
Epoch 246
Validation auc = 0.753521
Epoch 247
Validation auc = 0.744836
Epoch 248
Validation auc = 0.743662
Epoch 249
Loss = 1.0893e+00, PNorm = 94.0202, GNorm = 62.3417, lr_0 = 6.6056e-04
Validation auc = 0.732160
Epoch 250
Validation auc = 0.730047
Epoch 251
Validation auc = 0.723239
Epoch 252
Validation auc = 0.720892
Epoch 253
Validation auc = 0.715258
Epoch 254
Loss = 2.9257e+00, PNorm = 94.2855, GNorm = 114.7261, lr_0 = 6.5491e-04
Validation auc = 0.734038
Epoch 255
Validation auc = 0.742488
Epoch 256
Validation auc = 0.746479
Epoch 257
Validation auc = 0.742254
Epoch 258
Validation auc = 0.742488
Epoch 259
Loss = 1.2443e+00, PNorm = 94.6002, GNorm = 128.6489, lr_0 = 6.4931e-04
Validation auc = 0.738732
Epoch 260
Validation auc = 0.737793
Epoch 261
Validation auc = 0.740610
Epoch 262
Validation auc = 0.739437
Epoch 263
Validation auc = 0.740610
Epoch 264
Loss = 1.2092e+00, PNorm = 94.8995, GNorm = 97.8727, lr_0 = 6.4376e-04
Validation auc = 0.738028
Epoch 265
Validation auc = 0.746948
Epoch 266
Validation auc = 0.752347
Epoch 267
Validation auc = 0.753756
Epoch 268
Validation auc = 0.754225
Epoch 269
Loss = 1.6599e+00, PNorm = 95.1727, GNorm = 119.1859, lr_0 = 6.3826e-04
Validation auc = 0.751643
Epoch 270
Validation auc = 0.751408
Epoch 271
Validation auc = 0.744601
Epoch 272
Validation auc = 0.738967
Epoch 273
Validation auc = 0.728873
Epoch 274
Loss = 2.1662e+00, PNorm = 95.4645, GNorm = 185.6303, lr_0 = 6.3280e-04
Validation auc = 0.725117
Epoch 275
Validation auc = 0.734977
Epoch 276
Validation auc = 0.739906
Epoch 277
Validation auc = 0.743662
Epoch 278
Validation auc = 0.740141
Epoch 279
Loss = 1.5213e+00, PNorm = 95.8524, GNorm = 92.5371, lr_0 = 6.2739e-04
Validation auc = 0.744366
Epoch 280
Validation auc = 0.743897
Epoch 281
Validation auc = 0.753756
Epoch 282
Validation auc = 0.765493
Epoch 283
Validation auc = 0.774413
Epoch 284
Loss = 4.0974e+00, PNorm = 96.0786, GNorm = 254.1680, lr_0 = 6.2203e-04
Validation auc = 0.783803
Epoch 285
Validation auc = 0.788967
Epoch 286
Validation auc = 0.794601
Epoch 287
Validation auc = 0.799765
Epoch 288
Validation auc = 0.789906
Epoch 289
Loss = 2.4200e+00, PNorm = 96.3152, GNorm = 210.6470, lr_0 = 6.1671e-04
Validation auc = 0.787793
Epoch 290
Validation auc = 0.773239
Epoch 291
Validation auc = 0.765962
Epoch 292
Validation auc = 0.760798
Epoch 293
Validation auc = 0.753756
Epoch 294
Loss = 1.2987e+00, PNorm = 96.6102, GNorm = 255.5125, lr_0 = 6.1144e-04
Validation auc = 0.746244
Epoch 295
Validation auc = 0.750469
Epoch 296
Validation auc = 0.761737
Epoch 297
Validation auc = 0.768075
Epoch 298
Validation auc = 0.776056
Epoch 299
Loss = 1.9106e-01, PNorm = 96.8909, GNorm = 54.4902, lr_0 = 6.0621e-04
Validation auc = 0.778404
Model 0 best validation auc = 0.799765 on epoch 287
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.824202
Ensemble test auc = 0.824202
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.571831
Epoch 1
Validation auc = 0.730047
Epoch 2
Validation auc = 0.687559
Epoch 3
Validation auc = 0.729343
Epoch 4
Loss = 4.7600e+01, PNorm = 78.6871, GNorm = 1154.2831, lr_0 = 6.5000e-04
Validation auc = 0.736150
Epoch 5
Validation auc = 0.674883
Epoch 6
Validation auc = 0.669249
Epoch 7
Validation auc = 0.656103
Epoch 8
Validation auc = 0.686150
Epoch 9
Loss = 4.0242e+01, PNorm = 78.9914, GNorm = 1010.0066, lr_0 = 9.9743e-04
Validation auc = 0.679108
Epoch 10
Validation auc = 0.688263
Epoch 11
Validation auc = 0.708216
Epoch 12
Validation auc = 0.729343
Epoch 13
Validation auc = 0.724413
Epoch 14
Loss = 3.7644e+01, PNorm = 79.3354, GNorm = 1023.2929, lr_0 = 9.8890e-04
Validation auc = 0.709624
Epoch 15
Validation auc = 0.686150
Epoch 16
Validation auc = 0.713380
Epoch 17
Validation auc = 0.713146
Epoch 18
Validation auc = 0.738263
Epoch 19
Loss = 3.7365e+01, PNorm = 79.6019, GNorm = 1679.5791, lr_0 = 9.8045e-04
Validation auc = 0.694131
Epoch 20
Validation auc = 0.733099
Epoch 21
Validation auc = 0.683333
Epoch 22
Validation auc = 0.682160
Epoch 23
Validation auc = 0.726995
Epoch 24
Loss = 5.0916e+01, PNorm = 79.7900, GNorm = 783.7880, lr_0 = 9.7207e-04
Validation auc = 0.678169
Epoch 25
Validation auc = 0.561268
Epoch 26
Validation auc = 0.677230
Epoch 27
Validation auc = 0.702347
Epoch 28
Validation auc = 0.658685
Epoch 29
Loss = 2.4494e+01, PNorm = 79.9457, GNorm = 1012.8386, lr_0 = 9.6376e-04
Validation auc = 0.649765
Epoch 30
Validation auc = 0.670423
Epoch 31
Validation auc = 0.684507
Epoch 32
Validation auc = 0.665493
Epoch 33
Validation auc = 0.685211
Epoch 34
Loss = 4.3548e+01, PNorm = 80.0713, GNorm = 1565.9996, lr_0 = 9.5552e-04
Validation auc = 0.654930
Epoch 35
Validation auc = 0.667371
Epoch 36
Validation auc = 0.691784
Epoch 37
Validation auc = 0.665962
Epoch 38
Validation auc = 0.602582
Epoch 39
Loss = 4.0550e+01, PNorm = 80.2055, GNorm = 632.0797, lr_0 = 9.4735e-04
Validation auc = 0.728404
Epoch 40
Validation auc = 0.689202
Epoch 41
Validation auc = 0.693662
Epoch 42
Validation auc = 0.703286
Epoch 43
Validation auc = 0.707746
Epoch 44
Loss = 4.5033e+01, PNorm = 80.3265, GNorm = 1665.7511, lr_0 = 9.3925e-04
Validation auc = 0.700939
Epoch 45
Validation auc = 0.633568
Epoch 46
Validation auc = 0.577934
Epoch 47
Validation auc = 0.682160
Epoch 48
Validation auc = 0.702347
Epoch 49
Loss = 5.6393e+01, PNorm = 80.4625, GNorm = 1612.2387, lr_0 = 9.3122e-04
Validation auc = 0.747183
Epoch 50
Validation auc = 0.611268
Epoch 51
Validation auc = 0.638498
Epoch 52
Validation auc = 0.673239
Epoch 53
Validation auc = 0.699061
Epoch 54
Loss = 3.8888e+01, PNorm = 80.6479, GNorm = 812.5523, lr_0 = 9.2326e-04
Validation auc = 0.719718
Epoch 55
Validation auc = 0.733803
Epoch 56
Validation auc = 0.680047
Epoch 57
Validation auc = 0.680751
Epoch 58
Validation auc = 0.702347
Epoch 59
Loss = 1.7781e+01, PNorm = 80.8975, GNorm = 465.2753, lr_0 = 9.1537e-04
Validation auc = 0.720657
Epoch 60
Validation auc = 0.741784
Epoch 61
Validation auc = 0.704460
Epoch 62
Validation auc = 0.671362
Epoch 63
Validation auc = 0.712441
Epoch 64
Loss = 1.3677e+01, PNorm = 81.3160, GNorm = 579.8705, lr_0 = 9.0754e-04
Validation auc = 0.724413
Epoch 65
Validation auc = 0.709859
Epoch 66
Validation auc = 0.701408
Epoch 67
Validation auc = 0.697653
Epoch 68
Validation auc = 0.692019
Epoch 69
Loss = 1.0752e+01, PNorm = 81.8649, GNorm = 248.9738, lr_0 = 8.9978e-04
Validation auc = 0.704930
Epoch 70
Validation auc = 0.701878
Epoch 71
Validation auc = 0.654460
Epoch 72
Validation auc = 0.717136
Epoch 73
Validation auc = 0.705634
Epoch 74
Loss = 1.6358e+01, PNorm = 82.2838, GNorm = 683.5336, lr_0 = 8.9209e-04
Validation auc = 0.692019
Epoch 75
Validation auc = 0.669718
Epoch 76
Validation auc = 0.667136
Epoch 77
Validation auc = 0.725117
Epoch 78
Validation auc = 0.721127
Epoch 79
Loss = 1.3145e+01, PNorm = 82.7585, GNorm = 383.5758, lr_0 = 8.8447e-04
Validation auc = 0.704460
Epoch 80
Validation auc = 0.674178
Epoch 81
Validation auc = 0.692488
Epoch 82
Validation auc = 0.715962
Epoch 83
Validation auc = 0.734507
Epoch 84
Loss = 2.0145e+01, PNorm = 83.1499, GNorm = 763.9487, lr_0 = 8.7691e-04
Validation auc = 0.727700
Epoch 85
Validation auc = 0.689906
Epoch 86
Validation auc = 0.680516
Epoch 87
Validation auc = 0.669953
Epoch 88
Validation auc = 0.706103
Epoch 89
Loss = 2.1595e+01, PNorm = 83.4897, GNorm = 603.5196, lr_0 = 8.6941e-04
Validation auc = 0.665258
Epoch 90
Validation auc = 0.700235
Epoch 91
Validation auc = 0.644131
Epoch 92
Validation auc = 0.641784
Epoch 93
Validation auc = 0.704460
Epoch 94
Loss = 1.9257e+01, PNorm = 83.8862, GNorm = 519.0837, lr_0 = 8.6198e-04
Validation auc = 0.674178
Epoch 95
Validation auc = 0.668545
Epoch 96
Validation auc = 0.738967
Epoch 97
Validation auc = 0.668310
Epoch 98
Validation auc = 0.666197
Epoch 99
Loss = 3.0590e+01, PNorm = 84.2329, GNorm = 431.7747, lr_0 = 8.5461e-04
Validation auc = 0.705399
Epoch 100
Validation auc = 0.751174
Epoch 101
Validation auc = 0.731925
Epoch 102
Validation auc = 0.767136
Epoch 103
Validation auc = 0.713146
Epoch 104
Loss = 7.3175e+00, PNorm = 84.6361, GNorm = 346.7650, lr_0 = 8.4730e-04
Validation auc = 0.660094
Epoch 105
Validation auc = 0.662911
Epoch 106
Validation auc = 0.764554
Epoch 107
Validation auc = 0.746948
Epoch 108
Validation auc = 0.743192
Epoch 109
Loss = 1.1451e+01, PNorm = 85.1297, GNorm = 548.0015, lr_0 = 8.4006e-04
Validation auc = 0.769249
Epoch 110
Validation auc = 0.720892
Epoch 111
Validation auc = 0.697887
Epoch 112
Validation auc = 0.727230
Epoch 113
Validation auc = 0.759624
Epoch 114
Loss = 8.2033e+00, PNorm = 85.5624, GNorm = 402.1575, lr_0 = 8.3288e-04
Validation auc = 0.768310
Epoch 115
Validation auc = 0.766432
Epoch 116
Validation auc = 0.717606
Epoch 117
Validation auc = 0.689202
Epoch 118
Validation auc = 0.698357
Epoch 119
Loss = 9.8045e+00, PNorm = 86.0795, GNorm = 436.9180, lr_0 = 8.2576e-04
Validation auc = 0.746479
Epoch 120
Validation auc = 0.767371
Epoch 121
Validation auc = 0.765493
Epoch 122
Validation auc = 0.759155
Epoch 123
Validation auc = 0.743192
Epoch 124
Loss = 7.8385e+00, PNorm = 86.6498, GNorm = 308.1285, lr_0 = 8.1870e-04
Validation auc = 0.741784
Epoch 125
Validation auc = 0.750939
Epoch 126
Validation auc = 0.747653
Epoch 127
Validation auc = 0.748122
Epoch 128
Validation auc = 0.753756
Epoch 129
Loss = 4.5529e+00, PNorm = 87.0533, GNorm = 199.1428, lr_0 = 8.1170e-04
Validation auc = 0.734742
Epoch 130
Validation auc = 0.723239
Epoch 131
Validation auc = 0.710329
Epoch 132
Validation auc = 0.733099
Epoch 133
Validation auc = 0.752347
Epoch 134
Loss = 3.0008e+00, PNorm = 87.3587, GNorm = 247.9893, lr_0 = 8.0476e-04
Validation auc = 0.761737
Epoch 135
Validation auc = 0.764319
Epoch 136
Validation auc = 0.770657
Epoch 137
Validation auc = 0.780986
Epoch 138
Validation auc = 0.786854
Epoch 139
Loss = 5.4578e+00, PNorm = 87.8188, GNorm = 420.6651, lr_0 = 7.9788e-04
Validation auc = 0.778169
Epoch 140
Validation auc = 0.767840
Epoch 141
Validation auc = 0.764085
Epoch 142
Validation auc = 0.763850
Epoch 143
Validation auc = 0.766901
Epoch 144
Loss = 8.0721e+00, PNorm = 88.2309, GNorm = 654.4178, lr_0 = 7.9106e-04
Validation auc = 0.772300
Epoch 145
Validation auc = 0.772770
Epoch 146
Validation auc = 0.768075
Epoch 147
Validation auc = 0.764554
Epoch 148
Validation auc = 0.754695
Epoch 149
Loss = 3.2013e+00, PNorm = 88.6134, GNorm = 302.7899, lr_0 = 7.8430e-04
Validation auc = 0.762911
Epoch 150
Validation auc = 0.750939
Epoch 151
Validation auc = 0.737324
Epoch 152
Validation auc = 0.742254
Epoch 153
Validation auc = 0.740141
Epoch 154
Loss = 3.3972e+00, PNorm = 88.9867, GNorm = 154.9836, lr_0 = 7.7759e-04
Validation auc = 0.710094
Epoch 155
Validation auc = 0.706808
Epoch 156
Validation auc = 0.743897
Epoch 157
Validation auc = 0.741784
Epoch 158
Validation auc = 0.736854
Epoch 159
Loss = 3.7574e+00, PNorm = 89.4720, GNorm = 191.0640, lr_0 = 7.7095e-04
Validation auc = 0.736854
Epoch 160
Validation auc = 0.736150
Epoch 161
Validation auc = 0.730047
Epoch 162
Validation auc = 0.715258
Epoch 163
Validation auc = 0.727934
Epoch 164
Loss = 4.0203e+00, PNorm = 89.8961, GNorm = 369.0489, lr_0 = 7.6436e-04
Validation auc = 0.737793
Epoch 165
Validation auc = 0.741315
Epoch 166
Validation auc = 0.743427
Epoch 167
Validation auc = 0.735446
Epoch 168
Validation auc = 0.727700
Epoch 169
Loss = 2.7104e+00, PNorm = 90.1831, GNorm = 199.6495, lr_0 = 7.5782e-04
Validation auc = 0.714319
Epoch 170
Validation auc = 0.713146
Epoch 171
Validation auc = 0.708920
Epoch 172
Validation auc = 0.706103
Epoch 173
Validation auc = 0.712676
Epoch 174
Loss = 2.0255e+00, PNorm = 90.5024, GNorm = 203.4640, lr_0 = 7.5134e-04
Validation auc = 0.716197
Epoch 175
Validation auc = 0.711502
Epoch 176
Validation auc = 0.706103
Epoch 177
Validation auc = 0.709859
Epoch 178
Validation auc = 0.725587
Epoch 179
Loss = 1.8956e+00, PNorm = 90.8227, GNorm = 78.8720, lr_0 = 7.4492e-04
Validation auc = 0.741784
Epoch 180
Validation auc = 0.745070
Epoch 181
Validation auc = 0.744601
Epoch 182
Validation auc = 0.750939
Epoch 183
Validation auc = 0.730516
Epoch 184
Loss = 5.3804e+00, PNorm = 91.0499, GNorm = 221.9787, lr_0 = 7.3855e-04
Validation auc = 0.726291
Epoch 185
Validation auc = 0.724413
Epoch 186
Validation auc = 0.727465
Epoch 187
Validation auc = 0.726291
Epoch 188
Validation auc = 0.728638
Epoch 189
Loss = 6.0391e+00, PNorm = 91.3025, GNorm = 178.8867, lr_0 = 7.3224e-04
Validation auc = 0.741315
Epoch 190
Validation auc = 0.739906
Epoch 191
Validation auc = 0.744366
Epoch 192
Validation auc = 0.742019
Epoch 193
Validation auc = 0.738498
Epoch 194
Loss = 1.4038e+00, PNorm = 91.6236, GNorm = 69.8966, lr_0 = 7.2598e-04
Validation auc = 0.720188
Epoch 195
Validation auc = 0.708451
Epoch 196
Validation auc = 0.693662
Epoch 197
Validation auc = 0.684272
Epoch 198
Validation auc = 0.684977
Epoch 199
Loss = 3.0894e+00, PNorm = 91.9898, GNorm = 551.3304, lr_0 = 7.1977e-04
Validation auc = 0.688263
Epoch 200
Validation auc = 0.691315
Epoch 201
Validation auc = 0.692723
Epoch 202
Validation auc = 0.698122
Epoch 203
Validation auc = 0.699765
Epoch 204
Loss = 3.2315e+00, PNorm = 92.4152, GNorm = 367.2326, lr_0 = 7.1362e-04
Validation auc = 0.699531
Epoch 205
Validation auc = 0.706573
Epoch 206
Validation auc = 0.708920
Epoch 207
Validation auc = 0.709624
Epoch 208
Validation auc = 0.716432
Epoch 209
Loss = 2.3386e+00, PNorm = 92.8332, GNorm = 104.6959, lr_0 = 7.0752e-04
Validation auc = 0.734507
Epoch 210
Validation auc = 0.757042
Epoch 211
Validation auc = 0.758685
Epoch 212
Validation auc = 0.762441
Epoch 213
Validation auc = 0.751643
Epoch 214
Loss = 1.8371e+00, PNorm = 93.2134, GNorm = 217.1838, lr_0 = 7.0147e-04
Validation auc = 0.732629
Epoch 215
Validation auc = 0.735681
Epoch 216
Validation auc = 0.740141
Epoch 217
Validation auc = 0.742958
Epoch 218
Validation auc = 0.750235
Epoch 219
Loss = 2.0369e+00, PNorm = 93.5153, GNorm = 62.7103, lr_0 = 6.9548e-04
Validation auc = 0.748592
Epoch 220
Validation auc = 0.747418
Epoch 221
Validation auc = 0.739671
Epoch 222
Validation auc = 0.735446
Epoch 223
Validation auc = 0.737324
Epoch 224
Loss = 1.7406e+00, PNorm = 93.7759, GNorm = 72.6157, lr_0 = 6.8953e-04
Validation auc = 0.723239
Epoch 225
Validation auc = 0.704695
Epoch 226
Validation auc = 0.698122
Epoch 227
Validation auc = 0.710329
Epoch 228
Validation auc = 0.710563
Epoch 229
Loss = 1.6861e+00, PNorm = 93.9980, GNorm = 175.7955, lr_0 = 6.8364e-04
Validation auc = 0.715728
Epoch 230
Validation auc = 0.716901
Epoch 231
Validation auc = 0.717371
Epoch 232
Validation auc = 0.730516
Epoch 233
Validation auc = 0.737793
Epoch 234
Loss = 5.6799e-01, PNorm = 94.2073, GNorm = 174.5324, lr_0 = 6.7779e-04
Validation auc = 0.740376
Epoch 235
Validation auc = 0.746009
Epoch 236
Validation auc = 0.739671
Epoch 237
Validation auc = 0.739906
Epoch 238
Validation auc = 0.744601
Epoch 239
Loss = 2.2600e+00, PNorm = 94.4611, GNorm = 208.7058, lr_0 = 6.7200e-04
Validation auc = 0.736854
Epoch 240
Validation auc = 0.733099
Epoch 241
Validation auc = 0.722535
Epoch 242
Validation auc = 0.718310
Epoch 243
Validation auc = 0.709859
Epoch 244
Loss = 1.2707e+00, PNorm = 94.7075, GNorm = 92.9472, lr_0 = 6.6625e-04
Validation auc = 0.704225
Epoch 245
Validation auc = 0.700469
Epoch 246
Validation auc = 0.709624
Epoch 247
Validation auc = 0.703052
Epoch 248
Validation auc = 0.703286
Epoch 249
Loss = 2.3208e+00, PNorm = 94.9348, GNorm = 218.9553, lr_0 = 6.6056e-04
Validation auc = 0.708685
Epoch 250
Validation auc = 0.720423
Epoch 251
Validation auc = 0.723944
Epoch 252
Validation auc = 0.719484
Epoch 253
Validation auc = 0.719484
Epoch 254
Loss = 1.4605e+00, PNorm = 95.2360, GNorm = 190.8899, lr_0 = 6.5491e-04
Validation auc = 0.726291
Epoch 255
Validation auc = 0.736150
Epoch 256
Validation auc = 0.744366
Epoch 257
Validation auc = 0.746244
Epoch 258
Validation auc = 0.753756
Epoch 259
Loss = 8.0840e-01, PNorm = 95.5814, GNorm = 64.6388, lr_0 = 6.4931e-04
Validation auc = 0.750235
Epoch 260
Validation auc = 0.740376
Epoch 261
Validation auc = 0.729577
Epoch 262
Validation auc = 0.725352
Epoch 263
Validation auc = 0.726291
Epoch 264
Loss = 1.9420e+00, PNorm = 95.8924, GNorm = 126.6377, lr_0 = 6.4376e-04
Validation auc = 0.733803
Epoch 265
Validation auc = 0.735446
Epoch 266
Validation auc = 0.735446
Epoch 267
Validation auc = 0.739202
Epoch 268
Validation auc = 0.741080
Epoch 269
Loss = 2.0012e+00, PNorm = 96.1175, GNorm = 122.3473, lr_0 = 6.3826e-04
Validation auc = 0.743427
Epoch 270
Validation auc = 0.741315
Epoch 271
Validation auc = 0.738967
Epoch 272
Validation auc = 0.740376
Epoch 273
Validation auc = 0.742019
Epoch 274
Loss = 9.6935e-01, PNorm = 96.3391, GNorm = 107.8954, lr_0 = 6.3280e-04
Validation auc = 0.742958
Epoch 275
Validation auc = 0.743662
Epoch 276
Validation auc = 0.745775
Epoch 277
Validation auc = 0.745775
Epoch 278
Validation auc = 0.747183
Epoch 279
Loss = 6.3292e-01, PNorm = 96.5424, GNorm = 62.1311, lr_0 = 6.2739e-04
Validation auc = 0.744836
Epoch 280
Validation auc = 0.741549
Epoch 281
Validation auc = 0.734977
Epoch 282
Validation auc = 0.735681
Epoch 283
Validation auc = 0.737089
Epoch 284
Loss = 3.8858e-01, PNorm = 96.8091, GNorm = 125.4958, lr_0 = 6.2203e-04
Validation auc = 0.744601
Epoch 285
Validation auc = 0.742254
Epoch 286
Validation auc = 0.734742
Epoch 287
Validation auc = 0.736150
Epoch 288
Validation auc = 0.730986
Epoch 289
Loss = 1.0013e+00, PNorm = 97.0423, GNorm = 155.1598, lr_0 = 6.1671e-04
Validation auc = 0.729108
Epoch 290
Validation auc = 0.730986
Epoch 291
Validation auc = 0.729108
Epoch 292
Validation auc = 0.716432
Epoch 293
Validation auc = 0.707042
Epoch 294
Loss = 2.1351e+00, PNorm = 97.2514, GNorm = 109.1650, lr_0 = 6.1144e-04
Validation auc = 0.708451
Epoch 295
Validation auc = 0.713146
Epoch 296
Validation auc = 0.712441
Epoch 297
Validation auc = 0.724413
Epoch 298
Validation auc = 0.720188
Epoch 299
Loss = 5.6857e-01, PNorm = 97.4874, GNorm = 86.8162, lr_0 = 6.0621e-04
Validation auc = 0.708920
Model 0 best validation auc = 0.786854 on epoch 138
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.789284
Ensemble test auc = 0.789284
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN(
    (encoder): ModuleList(
      (0): MPNEncoder(
        (dropout_layer): Dropout(p=0.35000000000000003, inplace=False)
        (act_func): ReLU()
        (W_i): Linear(in_features=147, out_features=1900, bias=False)
        (W_h): Linear(in_features=1900, out_features=1900, bias=False)
        (W_o): Linear(in_features=2033, out_features=1900, bias=True)
      )
    )
  )
  (ffn): Sequential(
    (0): Dropout(p=0.35000000000000003, inplace=False)
    (1): Linear(in_features=2200, out_features=1900, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.35000000000000003, inplace=False)
    (4): Linear(in_features=1900, out_features=1, bias=True)
  )
)
Number of parameters = 11,937,701
Moving model to cuda
Epoch 0
Validation auc = 0.654225
Epoch 1
Validation auc = 0.661502
Epoch 2
Validation auc = 0.717606
Epoch 3
Validation auc = 0.734038
Epoch 4
Loss = 3.6519e+01, PNorm = 78.6889, GNorm = 1028.1844, lr_0 = 6.5000e-04
Validation auc = 0.679343
Epoch 5
Validation auc = 0.680751
Epoch 6
Validation auc = 0.769718
Epoch 7
Validation auc = 0.687793
Epoch 8
Validation auc = 0.653052
Epoch 9
Loss = 2.0737e+01, PNorm = 79.0009, GNorm = 436.0316, lr_0 = 9.9743e-04
Validation auc = 0.714554
Epoch 10
Validation auc = 0.697653
Epoch 11
Validation auc = 0.673944
Epoch 12
Validation auc = 0.695775
Epoch 13
Validation auc = 0.699765
Epoch 14
Loss = 4.7682e+01, PNorm = 79.3163, GNorm = 1145.8138, lr_0 = 9.8890e-04
Validation auc = 0.646009
Epoch 15
Validation auc = 0.621596
Epoch 16
Validation auc = 0.635446
Epoch 17
Validation auc = 0.686150
Epoch 18
Validation auc = 0.693897
Epoch 19
Loss = 4.2594e+01, PNorm = 79.5372, GNorm = 996.0873, lr_0 = 9.8045e-04
Validation auc = 0.590610
Epoch 20
Validation auc = 0.729108
Epoch 21
Validation auc = 0.689671
Epoch 22
Validation auc = 0.715728
Epoch 23
Validation auc = 0.637089
Epoch 24
Loss = 3.4371e+01, PNorm = 79.6924, GNorm = 650.3705, lr_0 = 9.7207e-04
Validation auc = 0.631455
Epoch 25
Validation auc = 0.742958
Epoch 26
Validation auc = 0.699061
Epoch 27
Validation auc = 0.719249
Epoch 28
Validation auc = 0.661033
Epoch 29
Loss = 3.7514e+01, PNorm = 79.8166, GNorm = 1168.1327, lr_0 = 9.6376e-04
Validation auc = 0.674413
Epoch 30
Validation auc = 0.700000
Epoch 31
Validation auc = 0.703991
Epoch 32
Validation auc = 0.699531
Epoch 33
Validation auc = 0.695070
Epoch 34
Loss = 2.1551e+01, PNorm = 79.9342, GNorm = 499.4841, lr_0 = 9.5552e-04
Validation auc = 0.716667
Epoch 35
Validation auc = 0.736385
Epoch 36
Validation auc = 0.709155
Epoch 37
Validation auc = 0.641315
Epoch 38
Validation auc = 0.704930
Epoch 39
Loss = 2.2966e+01, PNorm = 80.0663, GNorm = 1355.0176, lr_0 = 9.4735e-04
Validation auc = 0.731455
Epoch 40
Validation auc = 0.712207
Epoch 41
Validation auc = 0.623005
Epoch 42
Validation auc = 0.646009
Epoch 43
Validation auc = 0.700939
Epoch 44
Loss = 2.7736e+01, PNorm = 80.2436, GNorm = 530.5274, lr_0 = 9.3925e-04
Validation auc = 0.706103
Epoch 45
Validation auc = 0.676291
Epoch 46
Validation auc = 0.659155
Epoch 47
Validation auc = 0.681925
Epoch 48
Validation auc = 0.717371
Epoch 49
Loss = 2.3977e+01, PNorm = 80.4720, GNorm = 714.4732, lr_0 = 9.3122e-04
Validation auc = 0.713146
Epoch 50
Validation auc = 0.675117
Epoch 51
Validation auc = 0.648826
Epoch 52
Validation auc = 0.738967
Epoch 53
Validation auc = 0.735211
Epoch 54
Loss = 2.1401e+01, PNorm = 80.7598, GNorm = 436.8824, lr_0 = 9.2326e-04
Validation auc = 0.728404
Epoch 55
Validation auc = 0.685915
Epoch 56
Validation auc = 0.730282
Epoch 57
Validation auc = 0.704225
Epoch 58
Validation auc = 0.673239
Epoch 59
Loss = 3.1926e+01, PNorm = 81.0458, GNorm = 1255.8279, lr_0 = 9.1537e-04
Validation auc = 0.654930
Epoch 60
Validation auc = 0.707277
Epoch 61
Validation auc = 0.736385
Epoch 62
Validation auc = 0.645070
Epoch 63
Validation auc = 0.592958
Epoch 64
Loss = 3.6938e+01, PNorm = 81.4413, GNorm = 748.4976, lr_0 = 9.0754e-04
Validation auc = 0.709859
Epoch 65
Validation auc = 0.616432
Epoch 66
Validation auc = 0.745070
Epoch 67
Validation auc = 0.559390
Epoch 68
Validation auc = 0.562207
Epoch 69
Loss = 3.6531e+01, PNorm = 81.8676, GNorm = 634.9566, lr_0 = 8.9978e-04
Validation auc = 0.706103
Epoch 70
Validation auc = 0.760094
Epoch 71
Validation auc = 0.769953
Epoch 72
Validation auc = 0.722535
Epoch 73
Validation auc = 0.686620
Epoch 74
Loss = 1.2595e+01, PNorm = 82.3122, GNorm = 538.2337, lr_0 = 8.9209e-04
Validation auc = 0.676995
Epoch 75
Validation auc = 0.707746
Epoch 76
Validation auc = 0.749765
Epoch 77
Validation auc = 0.756573
Epoch 78
Validation auc = 0.712441
Epoch 79
Loss = 2.3092e+01, PNorm = 82.6919, GNorm = 1044.5515, lr_0 = 8.8447e-04
Validation auc = 0.727934
Epoch 80
Validation auc = 0.736150
Epoch 81
Validation auc = 0.715962
Epoch 82
Validation auc = 0.717136
Epoch 83
Validation auc = 0.687324
Epoch 84
Loss = 1.8326e+01, PNorm = 82.9733, GNorm = 902.9369, lr_0 = 8.7691e-04
Validation auc = 0.734507
Epoch 85
Validation auc = 0.712207
Epoch 86
Validation auc = 0.722770
Epoch 87
Validation auc = 0.674648
Epoch 88
Validation auc = 0.704695
Epoch 89
Loss = 7.1991e+00, PNorm = 83.2586, GNorm = 128.6791, lr_0 = 8.6941e-04
Validation auc = 0.743897
Epoch 90
Validation auc = 0.732160
Epoch 91
Validation auc = 0.733803
Epoch 92
Validation auc = 0.676995
Epoch 93
Validation auc = 0.647183
Epoch 94
Loss = 1.2895e+01, PNorm = 83.5412, GNorm = 650.5784, lr_0 = 8.6198e-04
Validation auc = 0.691549
Epoch 95
Validation auc = 0.723239
Epoch 96
Validation auc = 0.714319
Epoch 97
Validation auc = 0.704225
Epoch 98
Validation auc = 0.645305
Epoch 99
Loss = 1.4394e+01, PNorm = 83.9093, GNorm = 811.9125, lr_0 = 8.5461e-04
Validation auc = 0.653991
Epoch 100
Validation auc = 0.707981
Epoch 101
Validation auc = 0.719014
Epoch 102
Validation auc = 0.716667
Epoch 103
Validation auc = 0.711268
Epoch 104
Loss = 6.6726e+00, PNorm = 84.3920, GNorm = 180.9989, lr_0 = 8.4730e-04
Validation auc = 0.704460
Epoch 105
Validation auc = 0.713850
Epoch 106
Validation auc = 0.717371
Epoch 107
Validation auc = 0.737793
Epoch 108
Validation auc = 0.737559
Epoch 109
Loss = 1.3140e+01, PNorm = 84.7700, GNorm = 524.7678, lr_0 = 8.4006e-04
Validation auc = 0.742723
Epoch 110
Validation auc = 0.735915
Epoch 111
Validation auc = 0.722066
Epoch 112
Validation auc = 0.730047
Epoch 113
Validation auc = 0.747887
Epoch 114
Loss = 7.5695e+00, PNorm = 85.2130, GNorm = 635.9450, lr_0 = 8.3288e-04
Validation auc = 0.744366
Epoch 115
Validation auc = 0.739437
Epoch 116
Validation auc = 0.711737
Epoch 117
Validation auc = 0.704695
Epoch 118
Validation auc = 0.725352
Epoch 119
Loss = 4.5051e+00, PNorm = 85.6576, GNorm = 473.2407, lr_0 = 8.2576e-04
Validation auc = 0.732394
Epoch 120
Validation auc = 0.738028
Epoch 121
Validation auc = 0.735211
Epoch 122
Validation auc = 0.692019
Epoch 123
Validation auc = 0.680282
Epoch 124
Loss = 8.1932e+00, PNorm = 86.0427, GNorm = 478.2849, lr_0 = 8.1870e-04
Validation auc = 0.722770
Epoch 125
Validation auc = 0.714085
Epoch 126
Validation auc = 0.699061
Epoch 127
Validation auc = 0.706573
Epoch 128
Validation auc = 0.708920
Epoch 129
Loss = 1.1150e+01, PNorm = 86.3961, GNorm = 651.2338, lr_0 = 8.1170e-04
Validation auc = 0.683568
Epoch 130
Validation auc = 0.695305
Epoch 131
Validation auc = 0.735446
Epoch 132
Validation auc = 0.756103
Epoch 133
Validation auc = 0.756573
Epoch 134
Loss = 5.7648e+00, PNorm = 86.8176, GNorm = 173.6434, lr_0 = 8.0476e-04
Validation auc = 0.747887
Epoch 135
Validation auc = 0.739671
Epoch 136
Validation auc = 0.735915
Epoch 137
Validation auc = 0.745305
Epoch 138
Validation auc = 0.754460
Epoch 139
Loss = 6.9077e+00, PNorm = 87.3859, GNorm = 236.9771, lr_0 = 7.9788e-04
Validation auc = 0.753756
Epoch 140
Validation auc = 0.706808
Epoch 141
Validation auc = 0.696948
Epoch 142
Validation auc = 0.693192
Epoch 143
Validation auc = 0.720892
Epoch 144
Loss = 4.3373e+00, PNorm = 87.8152, GNorm = 331.4184, lr_0 = 7.9106e-04
Validation auc = 0.731221
Epoch 145
Validation auc = 0.736620
Epoch 146
Validation auc = 0.739437
Epoch 147
Validation auc = 0.728404
Epoch 148
Validation auc = 0.695540
Epoch 149
Loss = 5.2769e+00, PNorm = 88.1001, GNorm = 231.7746, lr_0 = 7.8430e-04
Validation auc = 0.677230
Epoch 150
Validation auc = 0.685681
Epoch 151
Validation auc = 0.706103
Epoch 152
Validation auc = 0.692723
Epoch 153
Validation auc = 0.662911
Epoch 154
Loss = 2.4278e+00, PNorm = 88.4711, GNorm = 41.6986, lr_0 = 7.7759e-04
Validation auc = 0.663380
Epoch 155
Validation auc = 0.658216
Epoch 156
Validation auc = 0.656338
Epoch 157
Validation auc = 0.655869
Epoch 158
Validation auc = 0.657277
Epoch 159
Loss = 3.7463e+00, PNorm = 88.7978, GNorm = 226.4055, lr_0 = 7.7095e-04
Validation auc = 0.658685
Epoch 160
Validation auc = 0.662911
Epoch 161
Validation auc = 0.675117
Epoch 162
Validation auc = 0.680516
Epoch 163
Validation auc = 0.693897
Epoch 164
Loss = 3.5820e+00, PNorm = 89.0879, GNorm = 405.2076, lr_0 = 7.6436e-04
Validation auc = 0.702817
Epoch 165
Validation auc = 0.690610
Epoch 166
Validation auc = 0.661737
Epoch 167
Validation auc = 0.646009
Epoch 168
Validation auc = 0.633333
Epoch 169
Loss = 4.7461e+00, PNorm = 89.4628, GNorm = 351.3612, lr_0 = 7.5782e-04
Validation auc = 0.647418
Epoch 170
Validation auc = 0.659624
Epoch 171
Validation auc = 0.660563
Epoch 172
Validation auc = 0.661033
Epoch 173
Validation auc = 0.652347
Epoch 174
Loss = 4.6328e+00, PNorm = 89.8230, GNorm = 221.6382, lr_0 = 7.5134e-04
Validation auc = 0.640610
Epoch 175
Validation auc = 0.634977
Epoch 176
Validation auc = 0.633568
Epoch 177
Validation auc = 0.651174
Epoch 178
Validation auc = 0.669249
Epoch 179
Loss = 6.1224e+00, PNorm = 90.2056, GNorm = 238.8570, lr_0 = 7.4492e-04
Validation auc = 0.668545
Epoch 180
Validation auc = 0.678873
Epoch 181
Validation auc = 0.685915
Epoch 182
Validation auc = 0.693897
Epoch 183
Validation auc = 0.699531
Epoch 184
Loss = 3.4595e+00, PNorm = 90.5099, GNorm = 204.5442, lr_0 = 7.3855e-04
Validation auc = 0.706808
Epoch 185
Validation auc = 0.714789
Epoch 186
Validation auc = 0.707512
Epoch 187
Validation auc = 0.682394
Epoch 188
Validation auc = 0.690610
Epoch 189
Loss = 7.4612e-01, PNorm = 90.8192, GNorm = 114.3050, lr_0 = 7.3224e-04
Validation auc = 0.698357
Epoch 190
Validation auc = 0.714085
Epoch 191
Validation auc = 0.728638
Epoch 192
Validation auc = 0.738967
Epoch 193
Validation auc = 0.737089
Epoch 194
Loss = 1.4611e+00, PNorm = 91.1054, GNorm = 220.7993, lr_0 = 7.2598e-04
Validation auc = 0.728638
Epoch 195
Validation auc = 0.738263
Epoch 196
Validation auc = 0.732864
Epoch 197
Validation auc = 0.715023
Epoch 198
Validation auc = 0.699061
Epoch 199
Loss = 1.8987e+00, PNorm = 91.3851, GNorm = 199.6205, lr_0 = 7.1977e-04
Validation auc = 0.707042
Epoch 200
Validation auc = 0.717840
Epoch 201
Validation auc = 0.731925
Epoch 202
Validation auc = 0.732864
Epoch 203
Validation auc = 0.739671
Epoch 204
Loss = 4.7368e+00, PNorm = 91.7928, GNorm = 355.0037, lr_0 = 7.1362e-04
Validation auc = 0.731925
Epoch 205
Validation auc = 0.706338
Epoch 206
Validation auc = 0.697887
Epoch 207
Validation auc = 0.703756
Epoch 208
Validation auc = 0.714789
Epoch 209
Loss = 1.4507e+00, PNorm = 92.1500, GNorm = 69.6584, lr_0 = 7.0752e-04
Validation auc = 0.720657
Epoch 210
Validation auc = 0.716667
Epoch 211
Validation auc = 0.739906
Epoch 212
Validation auc = 0.738967
Epoch 213
Validation auc = 0.740845
Epoch 214
Loss = 2.4860e+00, PNorm = 92.4355, GNorm = 124.7419, lr_0 = 7.0147e-04
Validation auc = 0.733333
Epoch 215
Validation auc = 0.732394
Epoch 216
Validation auc = 0.732160
Epoch 217
Validation auc = 0.737089
Epoch 218
Validation auc = 0.739202
Epoch 219
Loss = 2.0035e+00, PNorm = 92.7392, GNorm = 216.9618, lr_0 = 6.9548e-04
Validation auc = 0.734272
Epoch 220
Validation auc = 0.737793
Epoch 221
Validation auc = 0.737793
Epoch 222
Validation auc = 0.737793
Epoch 223
Validation auc = 0.734507
Epoch 224
Loss = 1.9852e+00, PNorm = 93.0893, GNorm = 141.0661, lr_0 = 6.8953e-04
Validation auc = 0.738498
Epoch 225
Validation auc = 0.741080
Epoch 226
Validation auc = 0.713146
Epoch 227
Validation auc = 0.715962
Epoch 228
Validation auc = 0.724178
Epoch 229
Loss = 1.6286e+00, PNorm = 93.5193, GNorm = 131.3951, lr_0 = 6.8364e-04
Validation auc = 0.723239
Epoch 230
Validation auc = 0.725352
Epoch 231
Validation auc = 0.722066
Epoch 232
Validation auc = 0.709155
Epoch 233
Validation auc = 0.707277
Epoch 234
Loss = 1.6016e+00, PNorm = 93.9009, GNorm = 118.0352, lr_0 = 6.7779e-04
Validation auc = 0.690845
Epoch 235
Validation auc = 0.678169
Epoch 236
Validation auc = 0.685211
Epoch 237
Validation auc = 0.681455
Epoch 238
Validation auc = 0.679343
Epoch 239
Loss = 1.9840e+00, PNorm = 94.1915, GNorm = 167.1914, lr_0 = 6.7200e-04
Validation auc = 0.673474
Epoch 240
Validation auc = 0.679577
Epoch 241
Validation auc = 0.682864
Epoch 242
Validation auc = 0.695775
Epoch 243
Validation auc = 0.707042
Epoch 244
Loss = 1.8261e+00, PNorm = 94.4211, GNorm = 177.5039, lr_0 = 6.6625e-04
Validation auc = 0.693192
Epoch 245
Validation auc = 0.685681
Epoch 246
Validation auc = 0.681455
Epoch 247
Validation auc = 0.688498
Epoch 248
Validation auc = 0.704225
Epoch 249
Loss = 1.2583e+00, PNorm = 94.6688, GNorm = 184.9345, lr_0 = 6.6056e-04
Validation auc = 0.717136
Epoch 250
Validation auc = 0.713615
Epoch 251
Validation auc = 0.710094
Epoch 252
Validation auc = 0.718310
Epoch 253
Validation auc = 0.723239
Epoch 254
Loss = 1.7652e+00, PNorm = 94.9740, GNorm = 261.2793, lr_0 = 6.5491e-04
Validation auc = 0.701878
Epoch 255
Validation auc = 0.689437
Epoch 256
Validation auc = 0.688732
Epoch 257
Validation auc = 0.689671
Epoch 258
Validation auc = 0.689906
Epoch 259
Loss = 1.8750e+00, PNorm = 95.2614, GNorm = 86.7392, lr_0 = 6.4931e-04
Validation auc = 0.702347
Epoch 260
Validation auc = 0.724883
Epoch 261
Validation auc = 0.724883
Epoch 262
Validation auc = 0.732394
Epoch 263
Validation auc = 0.735446
Epoch 264
Loss = 9.4553e-01, PNorm = 95.5257, GNorm = 65.7514, lr_0 = 6.4376e-04
Validation auc = 0.733333
Epoch 265
Validation auc = 0.729812
Epoch 266
Validation auc = 0.740610
Epoch 267
Validation auc = 0.744601
Epoch 268
Validation auc = 0.742488
Epoch 269
Loss = 1.4424e+00, PNorm = 95.7427, GNorm = 84.7873, lr_0 = 6.3826e-04
Validation auc = 0.738967
Epoch 270
Validation auc = 0.738967
Epoch 271
Validation auc = 0.734742
Epoch 272
Validation auc = 0.734272
Epoch 273
Validation auc = 0.730047
Epoch 274
Loss = 2.0641e+00, PNorm = 95.9523, GNorm = 161.8581, lr_0 = 6.3280e-04
Validation auc = 0.726056
Epoch 275
Validation auc = 0.720892
Epoch 276
Validation auc = 0.718545
Epoch 277
Validation auc = 0.724178
Epoch 278
Validation auc = 0.726761
Epoch 279
Loss = 1.1585e+00, PNorm = 96.1536, GNorm = 64.8369, lr_0 = 6.2739e-04
Validation auc = 0.728638
Epoch 280
Validation auc = 0.727465
Epoch 281
Validation auc = 0.726291
Epoch 282
Validation auc = 0.719953
Epoch 283
Validation auc = 0.718545
Epoch 284
Loss = 1.9520e+00, PNorm = 96.3733, GNorm = 144.9324, lr_0 = 6.2203e-04
Validation auc = 0.712441
Epoch 285
Validation auc = 0.707277
Epoch 286
Validation auc = 0.692723
Epoch 287
Validation auc = 0.687089
Epoch 288
Validation auc = 0.692488
Epoch 289
Loss = 8.1173e-01, PNorm = 96.5450, GNorm = 98.5660, lr_0 = 6.1671e-04
Validation auc = 0.691784
Epoch 290
Validation auc = 0.691784
Epoch 291
Validation auc = 0.696009
Epoch 292
Validation auc = 0.698592
Epoch 293
Validation auc = 0.703991
Epoch 294
Loss = 1.3265e+00, PNorm = 96.6932, GNorm = 127.0675, lr_0 = 6.1144e-04
Validation auc = 0.716432
Epoch 295
Validation auc = 0.718075
Epoch 296
Validation auc = 0.714085
Epoch 297
Validation auc = 0.704695
Epoch 298
Validation auc = 0.703286
Epoch 299
Loss = 9.7031e-01, PNorm = 96.8567, GNorm = 103.3805, lr_0 = 6.0621e-04
Validation auc = 0.709155
Model 0 best validation auc = 0.769953 on epoch 71
Loading pretrained parameter "encoder.encoder.0.cached_zero_vector".
Loading pretrained parameter "encoder.encoder.0.W_i.weight".
Loading pretrained parameter "encoder.encoder.0.W_h.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.weight".
Loading pretrained parameter "encoder.encoder.0.W_o.bias".
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Moving model to cuda
Model 0 test auc = 0.779651
Ensemble test auc = 0.779651
10-fold cross validation
	Seed 0 ==> test auc = 0.764299
	Seed 1 ==> test auc = 0.772426
	Seed 2 ==> test auc = 0.760084
	Seed 3 ==> test auc = 0.821493
	Seed 4 ==> test auc = 0.756773
	Seed 5 ==> test auc = 0.777544
	Seed 6 ==> test auc = 0.743227
	Seed 7 ==> test auc = 0.824202
	Seed 8 ==> test auc = 0.789284
	Seed 9 ==> test auc = 0.779651
Overall test auc = 0.778898 +/- 0.025195
Elapsed time = 0:08:35
